In September 2018 I embarked on a 6-week long project for the 2018 Hackaday Prize Competition where I designed and built a dedicated hardware MIDI controller for the Sugar Bytes Turnado audio FX software plugin. The project ended up becoming one of the top 20 projects of the competition’s Musical Instrument Challenge.
Building your own custom hardware controller for a specific piece of audio software is something you may want to do if you are finding that you’re not able to control your audio plugins or DAWs with existing hardware devices in your preferred way. Therefore in this article I’m going to highlight four of the main aspects that you’d need to consider if thinking about designing your own hardware controller for audio software, providing options and recommendations for the different ways it could be done based on my experiences developing the Turnado Hardware MIDI Controller.
This is not, however, a technical guide on building MIDI controllers; for this please see my tutorial on How to Build a Simple DIY USB MIDI Controller Using Teensy as well as the Turnado controller project log.
When it comes to designing any type of user interface such as the panel of a hardware controller, there are many general principles of interface design and interaction design that should be followed in order to create successful and usable user interfaces. Attempting to explain these principles is beyond the scope of this article, however I recommend checking out Larry Constantine’s Principles of User Interface Design as well as Don Norman’s Principles of Interaction Design which helped with the design aspects of my Turnado controller covered in sections 2-4 of this article.
The first important step to designing your own hardware controller for software is to make sure that the software in question, and specifically the parameters of the software you want to control, can be controlled by an external hardware source. When dealing with audio software this will almost always be done using the MIDI protocol; there may be less-common alternative options (such as Open Sound Control or HID) however for simplicity's sake lets assume MIDI is the only option. One way to verify MIDI compatibility is to consult the software’s user manual, looking for a section called something like ‘MIDI Implementation’, ‘MIDI control’ or simply just ‘MIDI’. Conveniently, most modern audio software is MIDI-controllable, however if your software is not then unfortunately it will be very difficult or even impossible to build a hardware controller for it.
Once you’ve verified that your software is MIDI-controllable, the next step is to find out the types of MIDI messages that the hardware controller will need to send to control the software. Most controls/parameters on audio software will use MIDI CC messages for external control, however some may use more advanced types of MIDI messages such as SysEx or NRPN messages. You may also want to investigate if your software has any MIDI output functionality for providing feedback back to the controller (e.g. for visually displaying software parameter values on the controller), if desired. To find out the MIDI implementation of your software, consult the ‘MIDI’ / ‘MIDI Implementation’ / ‘MIDI Control’ section of the user manual.
There are two main points to consider when deciding on the types of controls for your controller. The control and the way it behaves should be appropriate for the software parameter it is controlling. For example, if controlling an on-off button-type parameter then you probably want to use a button-type control rather than a slider or dial control.
The control should allow you to perform your preferred physical interaction with the software. For example, if you want to control a parameter by moving your hand up and down in the air then you’ll want to use some kind of proximity/distance sensor as the control.
Aspects of control types that you may want to think about regarding the above points include – discrete vs. continuous (e.g. button vs. slider); absolute vs. relative (e.g. potentiometer vs. rotary encoder), momentary vs. latching (e.g. push button vs. toggle switch); and touch vs. in-air. There are many different types of off-the-shelf controls you could use on your controller; from common controls such as push buttons, toggle switches, potentiometers and rotary encoders; to more advanced controls and sensors such as pressure sensors, dual-axis joysticks, infrared sensors and touch screens; all offering different ways of physically controlling software parameters. For more information on this from a music point of view I recommend checking out the ‘Control’ chapter of PUSH TURN MOVE by Kim Bjørn.
A good example of a dedicated hardware controller with a well-designed set of controls is the Pioneer DJ DDJ-SZ2 device (above) for controlling the Serato DJ Pro software. By featuring jog wheels, sliders, pads and dials it allows the user to control the software in an appropriate and preferred manner.
The main type of control I used for controlling Turnado's main parameters was a thumb joystick – a self-centering continuous control that allows 'momentary' two-way control – as I find when using Turnado to perform live audio manipulation it is usually done only in short bursts rather than constantly, where a control that 'springs back' is very useful. Also you are able to operate these types of controls with just one finger – compared to dials that usually need to two fingers – meaning you can operate multiple controls and control multiple parameters simultaneously with greater ease. For more information on the controls I used on the Turnado controller see here.
When it comes to designing the control layout of your controller the most important principles of Larry Constantine’s Principles of User Interface Design to consider are the ‘Structure’, ‘Simplicity’ and ‘Visibility’ principles.
Based on these principles, below are the reasons behind the control layout of my Turnado controller that will most likely be applicable to your own controller designs.
The control layout roughly matches that of the software interface so that the interface is familiar to Turnado users and therefore easy to understand. Related controls are grouped together – the eight ‘Turnado knob’ controls along the bottom, the ‘Dictator fader’ controls top-left, the preset controls top-right, and the LCD and it’s controls in the middle.
The position of the groups of controls are based on how often they will be used – e.g. the eight ‘Turnado knob’ controls will be used most so are placed at the front, whereas the preset controls will be used less so are placed at the top corner on the side away from my dominant hand. The eight ‘Turnado knob’ joysticks are aligned in a single line (rather than in two lines of four like on the software interface) so that it is easier to access all of them at the same time. The LCD is positioned top-centre so that it will less likely be covered by your hand or arm when using the controller. There are no freely-assignable or redundant controls.
For more information on control and interface layout principles from a music point of view I recommend checking out the ‘Layout’ chapter of PUSH TURN MOVE.
Visual feedback is very important in hardware controllers, as even though the user has moved away from the software interface they still need to easily see the values of the parameters they are controlling. Also‘Feedback’ is a principle of both the interface design and interaction design principles I referenced at the beginning of this article.
With some controls, such as potentiometers and toggle switches, you are able to tell the value of the control or software parameter by the physical orientation/state of the control. However if you decide to use controls that can’t physically show their value, such as rotary encoders or touch pads/sensors, you’ll most likely want your controller to include a way of visually displaying these control/parameter values. There are two main types of components you can use for providing visual feedback
LEDs are usually the cheaper and technically simpler option, allowing parameter values to be displayed using brightness, colour and animations. In most cases you’ll need one or more LEDs per parameter depending on the number of possible values, therefore they may not be the best option if needing to display many parameters or high-resolution parameters.
LCDs. These are usually the more expensive and more technically complex option, however as a single LCD can display virtually any arrangement of text, shapes and colours then you are able to display multiple parameter values on a single component, more clearly and in a lot more detail. LCDs can also come in the form of touch screens for providing a further input mechanism for controlling the software.
However as pointed out in the ‘MIDI Compatibility’ section of this article, if wanting to use LEDs or LCDs to display software parameter values then the software may need to include MIDI output functionality for sending MIDI messages back to the controller.
A good example of a dedicated hardware controller with well-designed visual feedback is the Native Instruments Komplete Kontrol S range of keyboards for controlling their Komplete software suite. They contain two LCDs for displaying lots of details about the software and parameters that are being controlled, backlit buttons for displaying parameter states, and LEDs above each key to show specific keyboard state and function information.
With the Turnado hardware MIDI controller I included a single LCD for displaying visual feedback, drawing virtual sliders to display the values of all controllable software parameters and using text to label each slider as well as to display settings of the controller (e.g. current MIDI channel). The values of the eight ‘Turnado knobs’ are updated via MIDI messages from Turnado, however the other displayed parameters are values from the internal state of the controller as Turnado cannot send MIDI messages for these particular parameters.