When solving problems for clients, I like to use worked examples to help me understand how well a particular technology might benefit the client.
You can have a live look at the PivotApp here. You can get a feel for the kinds of thing that the PivotApp does. I encourage you to give it a quick try before reading on. (If for some reason the link isn’t working, it probably means I’m doing some maintenance on it.)
Exercise your Data
Animate to See Outliers
The PivotApp leverages the D3 data-driven documents platform to create cool animated graphs. But this isn’t just animation for the sake of being cool; animation is great for seeing how outliers affect different aspects of the same data: a benefit sometimes called Object Constancy.
I Never Metadata DSL I Didn’t Like
The PivotApp has a module that encapsulates its business rules in a single JSON metadata object, using a mini-spec called a Domain-Specific Language, or DSL. This is really nice, for a couple of reasons:
- I want those business rules where I can see them! I don’t want them spread out all over the module, much less the entire app. Plus, I hate it when business rules are implemented by code. I’ve gotten a lot of mileage out of the concept of “data your app”–where possible, use data to make your app do something, not code.
- I want to be confident that I can apply a rule whenever I want. I can do this because my DSL is a general-purpose mini-language, and it doesn’t care which values I want to plot on a graph axis, for example.
MongoDB Is Nice Because It’s Fast
PivotApp uses MongoDB as its persistent store. It does this because it’s very useful to have a little experimental platform that gives us some performance data for comparison purposes.
For example, on the web, there are some interesting discussions about the relative performance aspects between the Hadoop ecosystem and MongoDB. By putting MongoDB in PivotApp, we can figure out if (say) Hadoop MapReduce is better/stronger/faster than MongoDB Aggregation Pipeline.
It’s all about verification and validation–looking before you leap! After all, both MongoDB and Hadoop (not to mention SQL-based stores) aren’t that easy to learn and deploy. In the best case, we do our own mini-analysis prior to committing lots of development resources or clustering hardware.
NodeJS Is Viable
A lot of shops have gone away from .Net and Java mid-tiers and moved to NodeJS. The PivotApp gives us some confidence that this strategy could be a winner.
I can tell you that when I use NodeJS, it’s hard for me to even tell if I’m writing server code or browser code. As Forrest Gump once said: “That’s good! One less thing.”
React/Redux and Functional Programming
PivotApp is built on React/Redux and adheres strictly to Functional Programming standards. By doing this, we can gather some further datapoints that tell us whether using pure functions is a good idea, or just a fad.
But even better: using FP lets us write more complete unit tests. Since everything is a function, it’s far easier for our test harness to mock the outside world (because there’s almost nothing in it ). Smaller unit tests, easier to write. That translates to faster and cheaper deliveries with less rework.
Experiments Are Critical
I’ll wrap this up with this claim: Any sufficiently complex problem needs some modeling and experimentation. The modeling should be “quick and dirty”–a tiny percentage of the overall cost of the software–but it should lower the project’s ignorance by giving us insight into where the hard problems really are.
This isn’t any different from other engineering disicplines, really. What’s strange is that there are so many software projects that just assume total knowledge up front.
Agile methodologies don’t solve this problem–they only give us rapid failure feedback (which isn’t bad, but it’s solving a different problem ).
You have to play in the problem space awhile before you can spot its boundaries and sharp corners!