Add new comment

Publish Date: 
Sun, 10/19/2014 - 14:18

[This post follows up two earlier posts: Beyond WorkingWiki: a project for general reproducible research on the web and More details on "Beyond WorkingWiki". This one also was originally an email prompted by questions from Matt.]

Where a wiki does server-side HTML generation - you submit markup and it gives you web pages - static website generation does it "compile-time" on your laptop before you push the final HTML to the web server. So this tool does the compilation there, rather than at the wiki server.

But we should be able to open it up a cloud sort of thing where you write the markup on your laptop and the WMD postprocessor farms out the computations to a server. We have an architecture that will do that pretty readily.

WorkingWiki has been designed in two layers: WorkingWiki, the upper layer, is a MediaWiki extension that processes these tags and other stuff stored in wikis, and displays files in the wiki's pages. When it needs to store files into a working directory, run make on them, and retrieve the output files, it does it by making requests to the other layer, ProjectEngine. ProjectEngine can be thought of as something like an HTTP server: you ask it for files and it serves them, the big difference being that it may run make to create the file before serving it.

We usually run WW and PE in the same process on the wiki's web server, but they can be separated onto different servers.

We can also have the WMD processor on the laptop send and request files to/from a PE instance somewhere. This could allow us to, for example, develop research material on a WW-enabled wiki, and then write a blog post about it on another system, that includes figures and text files reproducibly by pulling them directly from the wiki's project engine, rather than downloading images and including them in the blog post with the links to their origins broken.

It could also allow a self-publishing scientist to provide all the research project's code in a virtual machine image, furnished with all the right versions of R, python, etc., and ready to run a PE instance, and then the research is reproducible because you know it will run the code without compatibility problems. You could download the markup source of the blog post or wiki page to your computer, configured to re-run all the computations within that virtual project engine, and ready for you to "fork" the research and try out changes and alternatives by editing the source code and rerunning it.

I feel strange posting all this, since so much of it is speculation, but this is some of what I'm thinking about this project. Hope that makes sense...

Drupal theme by Kiwi Themes.