In this post, we discuss some of the problems and solutions of introducing domain-specific languages (DSLs) to store data in an existing application. But first, a brief recap. In the first post of this series, we explained why you might want to store some of your data in the form of text files written in a domain-specific language (DSL). Next, in the second post, we showed how some legacy applications can benefit from an intermediate solution where the data is already stored in text documents using JSON. Although this solution offers some version management advantages, it is primarily useful as a stepping stone in an incremental migration to a full DSL approach. In this third and final post, we discussed the full DSL approach to store data from an existing application, including typical problems and solutions.
Writing typecheckers is hard because a myriad of constraints must be checked. That leads to entangled and difficult-to-maintain typechecker code. TypePal is a new approach to improve upon this situation. We use it in many projects at Swat.engineering, including the typechecker of the open-source Bird DSL.
This is the second episode on replacing binary data with a textual representation. In the previous episode, we described the benefits of storing data in the form of a domain-specific language (DSL). Here, we will describe a convenient first step in this replacement process: the substitution of a legacy key/value database library with a drop-in replacement that stores the data in a textual format with one record per line. Besides being an ideal stepping stone for further migration, it also provides some basic version control functionality. That allows users in the organization to adjust to working with version control tools.
Following the process explained in our post Keeping Track of Requirements with Light-weight Tools will yield us a set of requirements. How can we turn these into a reliable roadmap for the future evolution of an application, team or project?
Syntax highlighting improves the productivity of DSL users. However, building a syntax highlighter is normally a serious investment and maintenance burden on the DSL developers. At Swat.engineering, we build Rascal-based syntax highlighters using a single-source-of-truth philosophy. This improves the synergy among language tools and simplifies their maintenance.
The tragedy of the commons is a well-known metaphor for a common good that is destroyed by citizens’ greed and lack of care. If all sheep of the village use the common meadow too often, the meadow will become dry and barren and will no longer feed any sheep. In a similar fashion, open-source software constitutes a gigantic software commons from which the whole world benefits: companies and citizens alike. How does Swat.engineering contribute to this software commons?
Writing and maintaining parsers for the binary data that surrounds us is a tough challenge. Enter Bird, a domain-specific language developed by Swat.engineering. It provides a declarative approach to describing a binary file format and fully automates the process of creating and maintaining a parser for it. Bird is an enabler for building tools for network and performance monitoring, cybersecurity, and digital forensics.
Any substantial change to critical software should be backed by a high confidence in correctness. Testing can be difficult when working with existing software systems. Automated tests might be sparse and hard to write effectively, while manual testing practices are often error-prone and labor-intensive.During the reverse-engineering of legacy systems, in particular, correctness can be challenging to define and components might interact in non-obvious ways. Therefore, simply “writing tests” is not as simple as it sounds.
I write a lot of Java. These days, I use Visual Studio Code. Sometimes, I see a variable but don’t remember its initial value. No worries: I just put the cursor on the variable, press F12, and watch the editor navigate to the declaration. It’s called “Go to Definition” in Visual Studio Code. Many other IDEs have it, too, and for many other languages. It’s a very useful feature.
Many legacy applications store data in binary files by using generic database systems like Microsoft Access, proprietary libraries or homegrown binary formats. All these methods have one thing in common: the data is stored in an opaque binary format that only the application can decipher.
Swat.engineering wants to help its customers deliver the right value. So we often start with a requirement analysis phase. This yields a requirement analysis report: How can we use the least amount of tools and overhead to focus on the essentials? We’ll show you how we use Google spreadsheets, Rascal, and HTML to generate interactive/traceable requirements. These requirements can not only guide a single software project but they can also be used to formulate a full project roadmap, see From Requirements to Roadmap (to be published).
“To a person with a hammer, every problem looks like a nail”. In other words, some people have a one-size-fits-all tool to solve everything. You might think that we’re guilty of this at Swat.engineering because we use Rascal a lot. So, are we myopic software engineers obsessed with a single language and its tools? Not at all. Here is our nuanced story.
Most IT systems consist of closely interacting software components (applications/tools) that solve related problems in the same domain.
Many components are based on the same domain elements.
What to do when a legacy system becomes too expensive to maintain or to add new functionality?
Rebuilding the legacy system from scratch will be expensive, error-prone and maintenance-intensive.
When we incorprated Swat.engineering early 2017, we quickly registered the domain swat.engineering and put there some temporary information.
Little did we know that “temporary” would mean 3 years.