Daffodils from Dials - or - Midi for great Real World Controllers
We had some good tech/art fun with a workshop at the Hope Center creating midi controlled, robot drawn, but human enhanced, algorithmic flowers :) This article is to share some of the tech in case it helps in your own projects. The robot flowers used a combination of a techniques - theres not space to cover everything here so we’ll focus on the midi controller part. Hopefully, the other pieces will get their turn soon :)
Overall Setup
The Midi Controlled, Human Augmented, Robot Drawn, Algorithmic Flowers (MCHARDAF) had the following overall setup:
Sixteen dials on a midi mixer for the input controls,
run into some browser based javascript to interactively create flower forms.
Once the human is happy, the data gets sent to a Python script,
that re-codes the data as gcode,
and sends it to a robot drawing machine.
And that, or another human takes the drawing as a starting point for more creating.
(It would have been more sensible to replace the python code with a bit more javascript running in the browser, and still from the browser communicate with the drawing robot. Its just that I had some existing Python code that I could re-use, so I went with the lazy, messy option!)
Im happy to say the project was very well received and much enjoyed. For me, the physical controls were absolutely key. The sense of connection between the tactile dial-turning experience and the instantanious creating on screen makes the digital object something real. The flower becomes a virtual material that you literallty touch with the controls. The difference between that and an on-screen slider or click box can be huge. I would prefer to swap out the word controller and say something more like point-of-contact or interface-between the human and the machine. You could think of it as the language the human uses to communicate to the machine. The range of possible controllers (thus in effect ‘languages’ ) is infinite. Which is an opportunity and a bit of a challenge. But solid, reliable, old school Midi can solve a few of the practical problems for you!
The Case for Getting Stuff Done in a Browser.
I’ve become a big fan of making things browser based where possible. If there’s a bit of code you want to run for whatever reason, consider running as Javascript in a browser! It seemed a bit of a fuss at first, but the advantages are undeniable. Especially the fact that your tech suddenly isn’t locked away inside your laptop under some socks. Its naturally inclined to be accessible worldwide at no extra effort. Javascript running in a browser is very fast in practice, there are great GPU accelerated graphics facilities just waiting there and the list goes on. One downside is that saving and loading files from browser based code isn’t very simple because of security. Surmountable though – best just to bite it and get on with the work in hand!
Communicating with the ‘Real’ World
They didn’t used to, but browsers these days have a number of ways they can communicate with real world devices. One very handy method is Midi. Midi is a ‘mature’ tech, being introduced in 1983, back when CD’s were cutting edge because the tape couldn’t get tangled. It was invented mostly to connect synthesisers together and thirty years later, browsers now talk midi too. Key takeaway: modern machines do not find 80’s tech very difficult!
The idea is super simple. If you have some sort of input device, whatever that may be: A key on a music keyboard, a twist dial, or something more exotic that measures the degree to which two hampsters are ignoring each other. It can send a midi message, which is essentially three numbers. In ‘standard’ situations, two of those numbers identify the sender and the third is the control value, but rules are made to be broken… A message can also just as well go from the computer to the device, if you have a reason for that.
Three Bytes (and not even that, really)
There’s a midi specification for what numbers mean what kind of messages. A certain range of numbers are reserved for the synth pitch wheel etc. If you’re using midi as a handy way to communicate data in some creative tech setup that will never see a standard synthesiser, then there’s really no need to worry about the proper rules!
A midi message is three bytes, except you only get to use 7 bits of each byte. In other words, the message is a set of three numbers, where each number is a integer from 0 to 127 inclusive. If you’re reading data from a standard device, you’ll likely see the first two numbers identify a specific button/key/control and the third indicate the state of that control. Personally, I just log the messages to the browser console, press buttons and look at whats happening. Below is a link to some code on github to help you do the same. If you’re making something unique then there’s nothing to stop you using the data any way you like :)
A little thing to consider
A lot of midi setups send messages only when something changes – like a button press or a dial twist. That means things almost certainly start out of sync when you power it all up. In that situation it can be handy to have some way for the software / browser side to request an update about where all the dials are - or to have a button on the controller that sends an update about where all the dials are. On my little Akai midi mixer, there’s a button marked ‘send all’ for exactly that reason. On a quirky custom device I made, I got it to send messages about the state of all its physical controls when it received a midi message to that effect.
Some Actual Code for Browser Midi Communication
I’ve put some simple code on github. You can find it here. There’s a weeny bit of HTML to hold it together and a fairly weeny bit of Javascript that actually does the work. To try it out you’ll need some kind of USB midi device – a little keyboard or whatever. Plug the device in, put the html and the js code in the same folder on your computer and open the html in a browser. Small note: I use Chrome and haven’t tested this code on many different browsers. If you use a different browser I cant say if you’ll need to adapt something slightly or not…
Open the browser console (f12 on Chrome) and read the reports the js code makes about the midi devices it can see. The code as written expects and selects a specific midi device by its name. Find your device name by reading the console reports then add that name into the code – the place is commented in the js file and hopefully makes sense! Or you could alter the code to be less picky and accept everything, or a million possibilities, of course…
With the code altered to include your hardware's name, reload the html, twiddle some dials / press some keys on your midi device and you should see the data logged to the console. Sorted :) The data and the world are yours to command.
Custom Hardware? Arduino is your friend
There are lots of kinds of Arduino. Those that use an ATmega32U4 chip can pretend to be any USB device you like – very handy! Examples are Arduino Pro Micro and Arduino Leonardo and others. You just use a USB Midi library for your Arduino and it will announce itself as a midi device. Add a bit of code to do whatever crazy data thing you have in mind, and what happens next is up to you! The custom devices in the photo at the top have all been made that way. I’ll cover some ideas and details in another article… Until then enjoy :)
Comments