The ideas behind Maia started life in 1999, while I was working on the design of a 3G radio receiver. To test my DSP code I wrote a VHDL library which read test vectors from a file and applied them to my DUT. Each line of the file contained a set of inputs, and a set of expected outputs following a clock edge:
[0, 1, abcd, 2345] -> [bcef0123, 45] # I/Os are hex constants
Sampled outputs were tested against the expected values. This was enough to create simple self-checking tests, and was a major step towards a fundamental goal: Separation of Concerns. When it worked, you could test any old or new functionality without writing any VHDL code at all: you just changed the vector file, and an automated test could tell you whether or not the RTL worked. I didn't even have to write the vector files myself - the algorithms group, or my boss, or anybody else, could create the file and test the RTL code. This was the 'Separation': I wrote the RTL, and anybody else could run the tests, and write new tests, and verify the code without knowing, or caring, about the details.
But, of course, the library had a number of major limitations:
- The testbench sequentially processed the vector file, with no ability to carry out branching or decision making. This meant that the test wasn't 'responsive', and there was no way to modify the test depending on the DUT state. In other words, DUT inputs couldn't be driven in response to an output.
- The vector file contained constant values for both the inputs to be driven, and the outputs to be tested. Everything therefore had to be known in advance, which made algorithmic testing impossible. For radio work this was generally sufficient, because the vector data was pre-computed by MATLAB anyway, but it made the code unusable in most other situations.
There were many other issues which, while not so important, effectively made the 'test vector' library unusable for general-purpose testing:
- The user still had to write some VHDL code to hard-wire clocks, signal names, widths, and so on
- It was difficult to test pipelines; additional code was needed to test expected values at some later point in the output sequence, and additional vectors had to be added for pipe setup and flush
- The code had fixed timing, and was only suitable for delta-delay simulations
- Everything was clocked: there was no combinatorial output testing
- There was no ability to wait for specific conditions on the DUT
- There was a single linear flow, so the test was single-threaded; multiple asynchronous interfaces couldn't be tested, for example
If you've ever been on a VHDL or Verilog course you'll recognise the test vector approach as Verification 101. As it happens, I had actually taught VHDL before starting on the 3G work. In one of the courses the students coded an alarm clock, and then had to verify it. The students had to write a testbench which read a set of commands ('opcodes') from a text file, and then applied them to the RTL. The commands set the alarm time, or set the current time, or advanced the current time, and so on. While it was a good way to learn VHDL there were, again, many issues. Code that was this specific and detailed could never be carried over to a new project, and the verification code would always have to be completely re-written, from scratch.
Both approaches - the test vectors, and the opcodes - are fundamentally about 'Separation of Concerns', even if we never specifically used that phrase in the course. It's the obvious thing to do, even if it's not obvious why it's obvious. However, both approaches ultimately fail, because they are fragile, and not portable. To address these issues, the two most basic requirements are:
- The addition of control constructs (looping, branching, decision making, and so on) around the 'vectors'
- The generalisation of the constant stimulus and test values to include a full expression syntax
The new 'vector file' therefore has to be language-based (even if it's just scripted), rather than being a list of simple vectors. And here's the problem: VHDL is not nearly sophisticated enough to process a 'vector file' of this complexity. The file must instead be compiled into something useful which can be simulated, and the compiler itself must be written in a suitable high-level general-purpose language (in other words, not VHDL or Verilog). The output of the compiler is the testbench (10), which can be simulated with a VHDL or Verilog simulator. Writing compilers is non-trivial, so I put the test vector problem on the back burner for a few years.
During 2003 I developed a configurable multi-issue processor, in SystemC, with SMART funding. This never became a commercial product, but as part of the process I built a verification system that tested the processor against a description of the required functionality (the instruction set, and what each instruction did). This included a compiler (written in C++) which turned the specification into Java-like byte code, which could be executed directly (SystemC is itself C++). The processor specification and test harness had a lot in common with what I wanted in a generalised test language, and this eventually became the starting point for Maia, and the mtv compiler. However, this took some years - I didn't get any more spare time until 2008, and the first usable mtv version appeared in 2009. mtv produced a Verilog testbench which could run directly on a simulator, rather than the original byte code. My original intention had been to write back-end code generators for both VHDL and Verilog, but dual-language simulators became a lot more affordable (or even free) at around this time, and the VHDL back-end has since disappeared from the priority list.
Maia has gone through a lot of development in the years since then, when I get breaks in my day job (I primarily do RTL development consultancy, and I've been working in Quantum Crypto for the past few years). There have been a number of releases since 2009, but these were essentially private, to people who were already using the language and were ready to put up with some significant limitations. mtv 2019.11 has a number of new features, and is the first release that I would consider to be ready for public use. The LRM has now reached v2.6, so there have been some changes in the language since the early days. There are still features to go in, and the odd bug to be fixed, so I'm expecting this to take a while yet.