“Interactor”, why did we built it?

Based on market feedback, Internet of Things (IoT/IoX) requirements originate from multiple sources which differ in perspectives. Bottlenecks arise because product expectations are not expressed consistently and quantitatively. They are fuzzy at the beginning and change throughout the development process. Additionally, IoT solutions are heavily dependent on many services to co-operate seamlessly. Lack of standards and relative in-experience of executing teams result in complicated and expensive solutions that become long-tail quickly.

Market insight dictates organizations to invest in “Abstraction” for productivity improvement. They allow for rapid low cost iterations that can remove bottlenecks and improve success ratios. We started testing out various productivity improvement models to enable rapid software development. We conducted various surveys, in person customer interactions, pain point identification and other lean-startup product-market analysis.

Studies indicated that, most of the customers wanted a graphical UI based programming tool and a set of dash-boards that could unleash their internal potential to create amazing outcomes. We found plethora of software products that precisely promised to suit their needs. Some of these products are built by prominent companies classified as enterprise middle-wares. They are also marketing extendable messaging layers to address scale in IoT use-cases.

Initially, we thought, it is a matter of educating users about these products and building smart overlays on top to create valuable solutions. Many companies have taken similar route. So, we started using existing technologies to build IoT products. We built using Java (back-end), Javascript (front-end), MQTT (messaging), JsPlumb (visual wiring), NoSQL (database) and supporting frameworks and libraries. We showcased our initial product at tech-crunch 2015 with moderate success. High-school students were able to create a working IoT solution within minutes using our product. Companies such as Cisco, Fujitsu, Panasonic showed elevated interest. But then we observed:

– Existing enterprise software technologies were not originally designed for IoT use-cases. – Even though graphical programming lowers the barrier of creating software solutions, it cannot be readily used to create and replicate comprehensive solutions implementing complex design patterns.

By delivering small-scale IoT projects, we realized that retrofitting existing software and consumption models to stitch various components together leads to sub-optimal solutions.Though past approaches create beneficial products, it does not necessarily translate to value that can be captured profitably. Thankfully, we are not alone in this realization. Many leaders are investing heavily in novel “Abstraction” methodologies to derive high productivity. However, majority are yet to discover this perspective:

– IoX is one of those cases where tweaking existing software assets will not result in leadership and it is evident as there is no clear market leader today. – Simply coding against a pub-sub messaging platform will not magically solve all problems.

We, then started figuring out ways to bring down the COGS (cost of producing) associated with IoT software solutions. We analyzed the reasons for high-cost among the solutions we built. Further, we conducted in-depth technical interviews with software engineers and architects. We arrived at the following hypothesis:

– Programmer productivity is dramatically lower when building server-side systems compared to user-interface systems. – Server-side logic cannot readily accomodate changes as it needs specialized skill-sets and in-depth knowledge of the code to make modifications. – The approach of a distributed light-weight client and heavy cloud based server drives up the TCO (cost of ownership) over lifetime of the solution.

The tests revealed to us, a broader persisting problem amongst back-end software solutions in the world. Most have difficulties with “threads and locks”. Even though, this limitation is an identified business risk, organizations continue to build concurrent software solutions with moderate engineering talent and absorb these risks in their costs. These organizations often deal with issues of software synchronization, deadlocks, complicated callbacks and variegated library layerings. All of these lead to expensive solutions.

Utilizing Javascript to build quick and cheap backend software using “*.js” platforms demanded a closer look. Also, the raising popularity of flow-based IoT platforms built on top of Node.js kindled our curiosity. It encouraged us to build solutions and test our market-derived notions. The single threaded event loop readily eliminated the issues of locks. Simpler logic were easy to build. However, issues emerged when we started to drill down to build complex scenarios. The system is pseudo-concurrent and turns out that we need more than JS expertise (C++) to achieve concurrent behavior. We discovered that one needs a myriad of libraries to build simple solutions and most of them stringify data. Further, for reasons unknown to us, async callback programming model is not natural to human brain. This defeats the purpose of building in-expensive backend logic with moderate engineers.

Summarizing our learnings from market experimentations, we find striking and consistent server-side requirements:

– Improve programmer productivity at minimal learning overhead. – Support synchronous and asynchronous execution models. – Minimal to no external library dependencies. – Embeddable and capable of running in memory constrained environments with comprehensive functionalities. – One distributed product irrespective of architecture (mist, fog, edge, cloud), to minimize deployment and operational risk. – Scalable both vertically and horizontally supporting flexible concurrency models.

Additionally, the need for backend software that can be programmed by technicians becomes apparent. When distributed solutions are rolled out by technicians and not experts, it becomes viable to deploy and maintain them at a large scale. This forces us to innovate a backend system for everyone to use, not only for experts.

To address the afore mentioned needs, we invented “The Interactor” by pursuing rigorous abstraction methodologies. Interactor’s technology is a clustered abstraction of semantically related layers of backend software. Interactor has the sophistications to change executing logic at runtime. We built it with good separation of concerns and balanced distribution of run-time responsibilities, yet focussed on visibility and simplicity. The relationships governing multiple services interacting with each other is managed transparently and efficiently.

Using Interactor software, we can create backend logic of any complexity without increasing work-load on individuals. It is consciously designed to eliminate dependencies between products built by different companies, that need to interact as part of the required IoX solution. Since, Interactor abstracts out repeatable server-side components of software and simplifies them into re-usable configurations, it enables architects to produce differentiated outcomes swiftly. Interactor’s in-built event and multi-processing support increases engineering productivity, streamlines software creation and aims to improve the operating margins. The productivity gain encourages us to re-distribute capital to develop additional capabilities to innovate.

Interactor is embeddable and we run it on linux based IoT gateways, cameras, raspberry pi for various solutions with complete capabilities. This permits us to build solution once and replicate it everywhere, from environments less than 100MB memory, all the way to large cloud servers. Solutions created using Interactor can be replicated through “copy and paste” to ease deployment process. Since our user-persona is a knowledgable software technician with no expertise, it requires minimal software skills to use and the configurations are easy to learn. Further, Interactor allows for multiple users to collaborate on a configured solution. Functional layers within the Interactor are discernible and debugabble independently. This enhances visibility, reduces error densities and provides us with control and confidence on delivered solution. In essence, we built it to scale technically and economically.

At Pulzze, we use Interactor as our backend engine to create custom solutions for our internal and customer needs. We host key aspects of our website, licensing and payment solutions on Interactor deployed in AWS instance. We are committed to continued investment in Interactor to derive more value from its benefits. We are thankful to the US government and The National Science Foundation for recognizing the value of our technology and funding our innovations.