Category Archives: Uncategorized

ReactJS, MongoDB and Blockly to Generate Data Entry Sites

I’ve been pulling together a proof of concept for some ideas I have about dynamic applications.  The gist is that dynamic languages (like JavaScript), combined with reactive designs (ReactJS/Flux) and with “schema-less” databases) allow a level of fluidity which makes generation of applications from metadata much more attractive.

These ideas have been put together into a production app for the company I currently work for.  The code I will share with you obviously will be a tiny subset of what we have done there, but I think it should be enough to prove to you that this approach has merits for specific types of applications.

The code for this blog is here:

Here’s an example of what the metadata might look like:

Blockly Metadata
Blockly Metadata





A few terms you will see in the metadata:

Use Case is the application you are building.  It contains namespaces which as you might guess define broad areas of the application in exactly the same way you would use them to define area of a code library.  Relations are, in essence, your objects. Namespaces also map directly to the databases you are storing in MongoDB.  Relations map to document collections in MongoDB.

How it works

Each of the blocks within the “Layout” section correspond to widgets created using React.  The widgets talk to each other using the Flux architecture.  Specifically, the app uses Redux.

The metadata itself is composed using the Blockly tool made by Google.  My understanding is that the Blockly tool is intended to be primarily a way to teach logic used in coding.  The blocks are used to generated code (such as JavaScript).  I’ve used them instead to generate JSON that contains metadata describing my application.

The information composed by these blocks is also used to generate a dynamic server side architecture that provides a REST API and writes to MongoDB.

I’ve tried to compose all these pieces using docker in a way that allows you to ignore the details of how everything is wired together.  The idea is to allow you to focus on what the code is doing.  That said, here are the details so you can dig in when you are ready (or if you need to debug a piece that is failing):

Here is what the above metadata generates:

Generated ReactJS App
Generated ReactJS App

High Level Pieces:

MongoDb (prefab docker container)

Database that is traditionally served on port 27017.   We will serve it from docker as localhost. If you want to change this, you will need to edit the mongoUrl constant in /server/src/config.js to use your own hostname and port

Nginx (prefab docker container)

Used to compose all the sites together on port 80 using reverse proxy.  If you don’t mind allowing CORS and pointing directly at port numbers, you can do without this.

Blockly Front End (your code)

This is where you define the metadata that drives your site.  This site is vanilla JavaScript, but uses some ES6 so you’ll want to run it using a modern browser.  I use Chrome (just regular not Canary).  This code is copied into the Nginx container to be served.

ReactJS App Front End (your code):

This requires ReactJS (obviously) which is not going to work well if you don’t run it through the provided NPM start script/WebPack configuration.  There are some things like ES6 modules and spread operators that aren’t supported in most browsers.  This code is copied into the Nginx container to be served.

Backend Server (your code)

This is your NodeJS REST API server.  It uses HapiJs because there aren’t many API focused frameworks that are simpler and yet (what I hope you’ll consider) feature complete.  This code is put into its own container and the Nginx reverse proxies to it.

Site Arrangement:

  • Blockly Front End: port 1350
  • ReactJS App Front End: port 3000
  • NodeJS Backend Server: port 10437

These are copied to or reverse proxied from nginx so that all are available on port 80 as follows:

  • Blockly Front End: /blockly
  • React App Front End (site root): /
  • Back End Server: /api

Nodegyp (or bcrypt) will not build on Windows

Nodegyp often fails to build/rebuild modules on Windows.  This happens to me every time I rebuild my Windows machine (and sometimes when I switch node versions).

Usually I attempt `npm install bcrypt` or similar and the log throws one of many errors related to node-gyp:

  • error: Can’t find “msbuild.exe”. Do you have Microsoft Visual Studio C++ 2008 installed?
  • LINK : fatal error LNK1181: cannot open input file ‘kernel32.lib’
  • msbuild.exe` failed with exit code: 1 (accompanied by something about nodegyp rebuild)
  • Failed to locate: “CL.exe”. The system cannot find the file specified.
  • and others…

Nodegyp is a way to abstract building native modules for NodeJs.   It is supposed to make things easier for cross-platform development using modules that have C or C++ code underlying them.

The problem is that Windows does not really have a simple tool chain for building native stuff.  With `nix systems, acquiring all the necessary tools is usually a single command line entry.  Windows is more a la carte due largely to the number of C++ runtimes that exist.

After a long and hideous season of pain, some folks tried to make some of this easier by creating the npm `windows-build-tools` package.  This will do quite a lot of the hard work of getting dependencies installed.  An alternative manual process is described here.

If this does not work, you may be forced to install Visual Studio 2013 Express.  This is pretty difficult to swallow for those of us who have already gone through the rather long download and install process of getting a newer version of Visual Studio running.

One thing you might try before going there is deleting your node cache.  This may seem like the nuclear option, but my experience is that dependencies can be hanging out in there, messing up the build stream.  My current guess is that this is caused by an install of Nodegyp or a Nodegyp dependency based on a different version of the C++ runtime or a different version of NodeJs.  Unfortunately doing a global uninstall of nodegyp did not fix the problem … so cache delete it was.

To blow away the cache, go to the %appdata% path (just drop that alias in your Windows Explorer path or `cd` to it in your command line), then delete the `npm` and `npm-cache` directories.

If the problem seems isolated to one project, consider blowing away node_modules and doing a fresh npm install.