wed, 01 aug 2007 10:45 - resonate

The Australian Music Center has a new journal ('web magazine'): resonate.

sat, 28 jul 2007 15:30 - java external for pd

Something I've often thought could be handy: a java plug in for pd. For simple externals this could be a lot simpler than wrestling with C/C++.

sat, 14 jul 2007 11:30 - msd & pmpd

Been working on a new idea lately - kind of a combination of string spheres and spinner where partials orbit around a fundamental mass - and each fundamental mass is linked to another. Not sure how it will work, but some useful googling turned up some hints on using tLinks in msd and also on combining msd and pmpd objects so you can use pmpd test objects for example.

sat, 23 jun 2007 11:30 - john whitney music box

A kind person at the creativity and cognition conference pointed me to the Whitney music box. Read about Whitney in wikipedia and also here.

tue, 17 apr 2007 11:30 - monograph: understanding interactivity

I found a nice conference paper written by Dag Svanaes and following up found his monograph Understanding Interactivity. Very relevant.

wed, 28 mar 2007 13:45 - book: trends in gestural control of music

Very useful online book: Trends in Gestural Control of Music edited by Marcelo M. Wanderley and Marc Battier. At first glance it appears to be a precursor to New Digital Musical Instruments: Control And Interaction Beyond the Keyboard.

tue, 06 mar 2007 10:50 - journal of usability studies

Linda posted a link to the Journal of Usability Studies, a new journal which I'd like to keep an eye on.

mon, 26 feb 2007 15:40 - nic collins' thesis, jared tarbell

From ACMA-L I heard that Nicholas Collins' PhD thesis, Towards Autonomous Agents for Live Computer Music: Listening and Interactive Music Systems is online. Promises to be an interesting read, and a good resource for how to structure a thesis in this area.

And, while I'm catching up on blogging, Andrew Brown suggested I check out Jared Tarbell, who has put together some musical visualisation works. The levitated.net site has a lot of information on these, although it's a little difficult to find some of it!

sat, 16 dec 2006 16:45 - note to self

When building stuff on Mac OS X, do the run "./configure" with the "--prefix=/sw" option. ie. The configure command should be "./configure --prefix=/sw". This keeps everything in the /sw directory instead of /usr/local or similar and is compatible with the way fink installs stuff (I think).

wed, 13 dec 2006 09:35 - mozart online

The complete works of Mozart are now available online. Barenreiter editions too it seems...

tue, 12 dec 2006 11:20 - bert bongers

We had a meeting this morning with Bert Bongers, who has done some very interesting work in the area of instrument design. He is visiting DAB at the moment and will present a seminar at 10.00am Thursday. His PhD thesis is available online. His book (based on the thesis I believe) can be purchased here.

sat, 09 dec 2006 10:45 - open dynamics engine

I notice that there is an open source library for simulating rigid body dynamics - the Open Dynamics Engine - , which could be the next (complicated) thing for me to work on now I've mastered (yeah, right) mass-spring systems.

sat, 09 dec 2006 16:00 - ben project notes

Some brainstormy ideas for Ben project discussed during my time in Brisbane...

• Allow different mics to control pitch/volume. This would mean that one musician could choose direction of force and another could control amount of force for example. Would also be possible for musicians to perform over a network. ie. In separate locations.
• Spheres as oracles... Not so sure about this one- spheres whisper text at times, triggered by..?
• Linking spheres - links between spheres are created/destroyed/altered dynamically so that shapes form and dissipate during the performance. This is my favourite idea at the moment. I can imagine some quite beautiful sounds and visuals coming out of this.

tue, 05 dec 2006 10:45 - notes from createworld

Stuart Cunningham gave a very interesting presentation on the somewhat scary Research Quality Framework (RQF). One key point for me was the need for people working in the 'creative cluster' to agree on terminology. He had a slide which showed a list of terms such as 'practice as research', 'practice-based research', etc. Another key point for me was that the RQF is an opportunity to address the lack of recognition for creative work in the previous DEST funding model. It seems that the UK and NZ experience with their RQF-style frameworks have been mixed but generally positive in this regard.

I like the idea of using podcasts in student assignments. Paul Draper & Matthew Hitchcok gave a presentation on their work doing this with music technology students. More complex in the subjects I teach. XP is probably the most obvious candidate for this but I worry about the increased workload for the students. I guess make it optional to either present to class or create podcast? In some ways I think this could be good for students with poor English, but on the other hand they may just see it as more 'busy work'. .NET is more difficult. Can't think of a smart way to use it...

The stuff that Andrew Brown, Andrew Sorenson and Steve Dillon are working on - the new version of Jam2Jam - looks really cool. The programming language that Andrew S has designed, Impromptu, is pretty amazing. Makes me feel like getting into Lisp - nice way to fill in all that spare time! Andrew B is keen to get the Spheres stuff running on their 'roadshow' where they get kids to play with the various toys they're designing. Would be a great opportunity to get some further investigation of the physical model interfaces with a different set of users.

Also, Steve Dillon mentioned a journal article, something along the lines of "Software Development as Research". Sounds very appropriate.

sat, 01 dec 2006 15:25 - aubio

Tim Boykett mentioned that aubio is worth a look for performing fiddle~ style onset/pitch detection, etc. I had come across it before but never tried it. Today I've been going through the usual compile/install process. Few notes:

• Downloaded aubio
• Used fink to install various required packages
• Manually compiled/installed libsamplerate which is not (yet) finkable. (for this I needed to also install libsndfile1-dev even though ./configure thought everything was ok.)
• Manually hacked the 'configure.ac' file in aubio main directory to replace all occurrences of "3.0.0" with "3.0". For some reason the configure script thought that version 3.0 of FFTW3 was an earlier version than 3.0.0 and wouldn't let me continue. After making this change I reran 'autoconf'.
• Now the config script tells me it can't find m_pd.h - the pure data header file. Even though it looks like it will compile without this, its important to have pd support. Next step- find this file...
• ok, found it (from maxlib source) and copied it into the /sw/include directory. Very ugly, but tricked the config script! Now, it's choking on compile. Maybe I should try to get it to compile sans PD first..?
• No good. Everything except PD plugins seem to compile. Next step: found this thread on the aubio discussion list. The suggestions produce a aubio.pd_darwin but it will not load when I run pd. Error: "[path...]extra/aubio.pd_darwin: dlcompat: dyld: [path]bin/pd can't open library: /usr/local/lib/libaubio.2.dylib (No such file or directory, errno = 2) aubio: can't load library"
• Ok, problem solved. The problem was that "sudo make install" never worked because make could never find a little program called 'mkinstalldirs' which is part of automake (I guess). Anyway, there was a bug in the file 'Makefile' in the /src /ext and /examples directories. It caused the make process to keep checking for 'mkinstalldirs' in ../ This was a problem because it wasn't there. Solution was to remove the "$(top_srcdir)/" from the following line: mkinstalldirs = $(SHELL) $(top_srcdir)/mkinstalldirs Ater doing this in all 3 directories, the pure data plugin seemed to work.

sat, 18 nov 2006 10:45 - c# text

Just a note to self: the book, Programming C#, 4th Edition available on Safari tech books online via the UTS Library looks a promising text for next semester.

thu, 09 nov 2006 09:00 - virtual orchestra

Just came across the Virtual Orchestra project at the GameLab in Nanyang Technological University in Singapore. Uses gesture recognition to allow users to virtually conduct an ensemble. Seems like tempo and dynamics are mapped to various gestures. Not sure how more subtle effects (articulation style, etc) might be controlled.

fri, 27 oct 2006 12:15 - some notes from DIME-ARTS 2006

Some quicky notes on the fly from DIME. Maywa Denki - a art/comic/parody.

Improvised Empathetic Device

Media Arts Festival Japan (DIME 2007?)

thu, 19 oct 2006 15:00 - preparing video for transana on mac

The best/only way to get from mini-dv to mpeg1 (or probably mpeg2 but I've not tried this yet) for use with Transana is to:

1. Open iMovie and import the clip you want to analyse. Important: Use a local disk to save the project - external drives are problematic. (Lots of bugs in iMovie 3 apparently.)
2. 'Share' the movie; choose File -> Share, Quicktime and export as a full DV file (with extension .dv).
3. Use the ffmpegX program to convert from DV format to mpeg1 format. The format is called VCD (Video CD) in ffmpgX. Note, in the 'options' tab, deselect 'split mpg' and 'Author as' options so you just end up with one big .mpg file. You can use either the mpeg2enc or ffmpeg encoders - as far as I know it makes no difference. (I've definitely used mpeg2enc and it's worked.)

EDIT (1/3/7): You can skip step 2 above - just use iMovie to import the clips from tape, then exit (after saving). This produces a .mov file which has separate files for each clip that it contains, but this doesn't matter. ffmpegX can read this file and can convert to mpg1 directly - saves heaps of time! Remember to deselect the 'split mpg' and 'Author as' options.

Once the above is complete - and it can take a while - you have an mpeg1 file which Transana can use without any trouble.

wed, 18 oct 2006 09:25 - interactive music systems

Just noticed that the Robert Rowe book, Interactive Music Systems is available as an 'ebook' from the UTS Library. This is a very widely cited book in this area.

tue, 10 oct 2006 11:25 - video analysis/annotation tools

Transana and Quilldriver both look interesting.

sat, 07 oct 2006 11:00 - long time between blogs: ixi

Well I've been slack in the blog department lately. Mostly I've been bogged down reading methodology stuff, which is kind of fun but also seems to have slowed down 'new work' stuff. Anyway, the main thing I wanted to say was I found the ixi website, which has lots of info about some very interesting sounding projects in the area of interfaces for musical expression. There are a bunch of papers, etc there that I've just printed - will report back later. I like the way they are presenting their work. I've been thinking about whether/how to make our musical toys more broadly available, both for getting feedback and perhaps encouraging others to extend them. I think that open sourcing them could be a cool way of going about this.

thu, 31 aug 2006 17:00 - goethe, kandinksy and synaesthesia

I borrowed Nicholas Cook's analysing musical multimedia from the library. In a section of the first chapter there is a great quote from Goethe on the relationship between sound and colour (p.46):

"Colour and sound do not admit to being compared together in any way, but both are referable to a higher formula, both are derivable, though each for itself, from this higher law. They are like two rivers which have their source in one and the same mountain, but consequently pursue their way under totally different conditions in two totally different regions, so that throughout the whole course of both no two points can be compared."

Naively, I agree. (In googling this I also came across Niels Hutchison's page on colour music, which appears to be a comprehensive look at this topic.) Cook goes on to point out Kandinsky's application of this basic approach in his work: "Kandinsky was much influenced by theosophical thought, and the starting-point for his theory is that the universe consists essentially of the play of vibrations and that these vibrations have a fundamentally spiritual significance." (p. 46) Later on the same page:

"Another way to visualize this would be a triangle, with Kandinsky's 'spiritual' (Goethe's 'higher law') at the apex; understood this way, sound and colour do not relate directly to one another, but relate indirectly through a common relationship with the spiritual. There is no question, then, of mapping the structures of sound and colour on to one another, in the manner of Skriabin's mapping of the colour wheel onto the circle of fifths. Instead, sound and colour correspond to one another in so far as they embody the same ultimate meaning."

thu, 31 aug 2006 12:20 - Practice-based research

Brigid pointed out a special issue of an journal called "Lier en Boog" on practice-based research. I'm not sure what the title means but the subtitle is Artistic Research. She mentioned that the article in that issue (vol 18) by Mika Hannula (p.70) is worth a look.

fri, 25 aug 2006 13:50 - Schoen

Been reading 'The Reflective Practitioner' by Donald Schoen (need an umlaut on the o but don't know how to get it without wasting time). Very useful description of 3 types of experiments:

1. Hypothesis testing - controlled experiment a la "traditional science" approach.
Exploratory experiment - Action is taken to see 'what happens'. Examples: infant exploring the world by doing stuff and seeing what happens; an artist juxtaposing colours to see what the effect is. (p. 145)
2. Move-testing experiment - Action is taken with the intention to achieve a certain aim. Action is affirmed if it achieves this aim and negated if not. In more complex situations the aim might need to be evaluated too. Example: You give your daughter a dollar to stop crying. It works, but long-term outcome is that she uses crying to make money. Therefore the question becomes not only, 'Did you get what you intended?' but also 'Do you like what you got?'. (p.146)

thu, 17 aug 2006 11:15 - brainstorm

I had an idea for another movement of the partial reflections piece (late) last night, or maybe a more teaching-focused toy. Sound floats a balloon around the screen. Uneven, jerky crescendos cause it to burst. Light, floaty playing levitates it around the screen. I think I'd like to play with something like that.

Playing different notes spawns more balloons??

thu, 17 aug 2006 10:25 - events

It's been pretty hectic lately. Ben and I did two performances of the spheres piece (now known as 'Partial Reflections') with Ensemble Offspring at Wollongong and also the Opera House Studio. I'd forgotten how full-on performing (and performers) can be and it was both cool and slightly unsettling to be there. The concert went well, although I was a little frustrated because the spinner piece had some 'clickiness' in the audio - due to the laptop processor not being able to quite keep up when things got busy I think. The studio's sound system, being top quality, made this more apparent, at least to my ears. The first movement (spheres) went really well in the studio. I managed to get the right level of spookiness in the sounds from the beginning, which has been an issue in the past.

Damien got good crowds to both performances, and we had a little mention in the Sydney Morning Herald, although no reviews as yet. Apparently someone from RealTime was there so there could be one shortly.

Also of course, we had the install and launch of the spheres-in-a-circle piece (confusingly known as 'Spheres of Influence') at beta_space in the Powerhouse Museum. I was a bit apprehensive about the launch - not feeling totally comfortable with art lingo - but on the night it was really enjoyable experience. I'm always surprised by just how much having to present your work to others helps clarify just what you're on about- to me anyway.

This might be a good place to say a big thankyou to Deborah, Matthew Connel and Greg for doing so much work to get everything set up, publicised and generally looking tickety-boo. And of course all the CCS volunteers who came along and helped out on the night- especially Roman for doing the video. Thanks!

thu, 27 jul 2006 10:25 - spirometer

Once again, thanks to Greg for providing info about Vernier's spirometer which George Khut is using as a kind of super sensitive midi breath-controller. Some extra bits are needed such as a serial-usb converter and a labpro. Would be interesting to hook it up to the end of brass instruments to give feedback on how much air at what speed is being used.

thu, 20 jul 2006 11:40 - breath car and rehearsal joypads

Greg Turner sent me a link from we make money not art about the breath car by Matthew Irvine Brown, designed to help students gain the breath control skills needed for singing/playing wind instruments. The student blows into a midi breath controller and can drive a toy car around by doing various things with their breath (blow harder to go right, etc). Could be a fun addition to the toolbox.

The article also links to the rehearsal joypads, which are mock-ups of instruments that connect to software that gives feedback on correct fingering/slide position, etc. When you play phrases correctly an on-screen band marches in time and stays together. If you don't they walk off in disgust! This could be great for beginner students...

wed, 28 jun 2006 14:15 - music education research

The latest Music Education Research journal (vol. 8, no. 2) has a number of articles on methods and methodologies in music education research. Looks pretty handy.

thu, 15 jun 2006 11:25 - rootkit!

Matt Waples (ex-UTS and sometime lecturer) has a mock-rock entry in google idol. Very cheesy and deserving of more votes!. They are the only Australian entry in the "original" category.

mon, 05 jun 2006 15:25 - electroplankton

Thanks to Julien Phalip for pointing me to electroplankton, a video game for the Nintendo DS which sounds pretty cool. The game provides a series of underwater "sound creatures" that you mess around with to create interesting sounds and visuals. It's not really a 'game' in the traditional sense but more an environment for creating interactive envrionments to explore musically. Some of the creatures let you record fragments of your voice which are played back at later stages, others make noises as they bounce around a virtual environment that you create. It's a cool idea.. I wonder if there is a big market for this kind of thing?

wed, 24 may 2006 20:00 - lemur

Tony Gould just tipped me off about the lemur, a souped-up controller for real-time control of music applications. It's a touch-screen tablet-like device which allows you to set up your own interfaces for controlling just about any parameter you like. An aspect which is especially interesting to me is the fact that you can assign physical properties to these virtual controllers and specify forces which act on them. Looks like a great device for live performance.

wed, 24 may 2006 09:30 - experimenta call for works

Mike forwarded a call for applications from experimenta. Up to $5,000 for works for an exhibit on next year. Here is the application form (pdf).

sat, 13 may 2006 14:20 - 'scanned synthesis'

Spending my usual Saturday playing with and reading about physical models, I found some discussion of scanned synthesis. Scanned synthesis uses mass-spring models to generate waveforms for sounds. Unlike the other physical modeling approaches I've blogged previously, scanned synthesis uses very low-frequency models (<15Hz). In the more hard-core physical modeling, an attempt is made to completely model some physical instrument- a violin for example - and once the model is constructed the virtual instrument is 'played'. All the physical events that would have occurred in the real violin now occur virtually in the model, and it outputs sounds that are (depending on the accuracy of the model) the same as the real one.

From what I gather, scanned synthesis generally uses simpler models updated at much lower frequencies. The models don't produce sound directly, but are 'scanned' at regular intervals to produce the waveforms. The idea is that a performer can manipulate the models in performance to produce musical/natural-sounding effects. From the above site:

"The psychophysical basis for Scanned Synthesis comes from our knowledge about human auditory perception and human motor control abilities. In the 1960's Risset showed that the spectra of interesting timbres must change with time. We observe that musically interesting change rates are less than about 15 hz which is also the rate humans can move their bodies. We have named these rates Haptic rates."

What I find interesting is that in my work I've been using live instruments to affect a low-frequency physical model, which in turn produces audio. I think this is an extension of scanned synthesis that I have not come across before - but that's not to say it's not been done!

Later: paper describing scanned synthesis is here.

wed, 3 may 2006 16:30 - ben notes

Up in Brisbane for a couple of days. Ben has been messing around with the spinners. Some comments: can balls be linked to the center, fixed mass? Preferably via a kind of stocking or sock-style link. eg. Instead of spinning out in orbit, the middle mass kind of elongates (bit like a sun-spot maybe?). Could spinners slow down gradually (ie. introduce friction?).

tue, 2 may 2006 17:30 - msd resource

At last I found a useful home page for msd!

sat, 29 apr 2006 16:00 - msd: physical modeling for pure data

I realise it's been a long time between blogs and I'm not really sure why. Just got out of the habit for a while I guess. Anyway, the motivation to blog again was provided by my (re)discovery of the msd externals for pure data. MSD (Mass-Spring-Damper) is a library for pure data that provides objects that take care of all the tedious calculations involved in creating models that respond in physically plausible ways. I have been using the pmpd libraries prior to this, but msd (which from what I can gather is an extension of pmpd) is much simpler/nicer to use. There is also a nice editor provided which makes it easy to try out some prototype models without spending hours creating them manually.

mon, 17 apr 2006 19:00 - latex/bibtex

I've been getting the hang of latex and I think I want to use this for my thesis, papers, etc. I'm practicing by writing the paper for DIME2006 using it. It's quite good, but citing references in Harvard format is a little tricky. This tutorial seems to explain how to do it...

thu, 13 apr 2006 16:35 - reminder!

Don't forget to collect the Terry Winograd book from the library: Bringing design to software (005.12 WINO). Greg mentions this in his methodology chapter. A substantial amount of it seems to be online here, but there are large chunks missing?

sat, 01 apr 2006 10:35 - physical model of brass instruments

I've been googling around physical modelling stuff for a while and came across Arturia, who have released software called Brass. Brass includes 'software recreations' of trumpet, trombone and saxophone (I guess no-one told the marketing department that sax is not a brass instrument) which take a different approach from the more traditional sample-based synthesisers. Instead of recording someone playing a real trumpet and then playing that recording back at different pitches (a simplfied description of how traditional wavetable synthesis works), this software actually creates a software model of a real trumpet which can then be 'played' by specifying some of the very large number of parameters that affect the sound quality. Parameters include attack, pressure, vibrato, mute and many more.

This approach offers a lot of interesting possibilities. For example, you could specify that your virtual trumpet be made of wood (and Brass apparently lets you do this), or make a virtual trombone out of play-doh.

I find it curious though that these types of products are often marketed as, "Now you have all the control of the experts - but without the need for years of training!" The problem is, of course, that the more accurate your model the closer the amount of training required will approach that of the real thing. Oh well, it's only marketing anyway.

The web page also has a description of some of the research behind Brass which was conducted with IRCAM.

Later: Online book by the guru of waveguide synthesis, Julius O. Smith III - Physical Audio Signal Processing For Virtual Musical Instruments and Audio Effects.

Still later: The Computer Music Journal has a great article by Ali Momeni and Cyrille Henry: "Dynamic Independent Mapping Layers for Concurrent Control of Audio and Video Synthesis". Dry title, but fantastic overview of the possibilities of using physical models to mediate between musicians, visuals and sound. These are the guys who developed the Physical Modelling for Pure Data objects I've been using. The article also mentions another set of objects which may be better for really large mass-spring-damper systems: MSD.

thu, 30 mar 2006 16:20 - paper brainstorm

A bit of paper planning.

Design and Evaluation of Interactive Music Toys

Probably a journal article, basically my methodology chapter. Should have a talk with Linda about this.

Circle Things (or whatever): An Interactive Toy for Musicians

A conference paper, but not sure where to target. Describes the process of designing the circle-spheres toy (ie. moving from Spheres piece to more 'grounded' tool/toy). Would include: rationale; process; past work; description of how sound is analysed and mapped to model; description of the toy itself; perhaps a technical/architectural description (?); Evaluation. The evaluation component is the missing element at the moment.

Spinner: Interactive Musical Work

Another conference paper in the same style as the IE 2005 one, this time describing the development of the spinner movement. Structure: rationale; past work; approach; the work itself; future work. Actually, if we included a detailed description of the music that Ben has written/is writing then this could perhaps be submitted to a music conference. (or maybe split this into two papers, one focused on technology and the other on the music?)

Using Real-time Physical Modelling to Reveal Characteristics of Musical Articulation

Less clear about this one. A paper which outlines how and why physical modelling can help make musical qualities visible. Describes how music is often described in physical terms (and even some musical theories which assign 'weights' and 'momentum' to various musical elements); show how our deep seated understanding of the world in physical terms can be used to aid musical learning using this approach; musical input is pathway from sound pressure levels, via physical movement of the microphone membrane to stream of numbers. This stream can then be used to map the physical microphone movement to other kinds of (psuedo) physical movement.

Using Vocal/Speech Analysis Techniques with Real Time Musical Input

This might be co-authored with James depending on how his project works out and could include description of using speech recognition techniques to categorise different types of articulation.

A Repertory Grid Study of Expert Musicians

Using Jankowicz's bootstrapping technique, analyse the results of the repertory grid interviews. I would say that I need to do about another 5-6 interviews first though.

tue, 28 mar 2006 11:05 - some grant possibilities

Seeing as we missed out on the Australia Council grant we applied for, I went looking this morning for some other sources of funds. I found a grants database that UTS has access to: SPIN. SPIN lets you search a huge grants database for various keywords. Two which at first glance appear worth a look are the Ian Potter foundation and the NSW Ministry for the Arts music grants.

mon, 27 mar 2006 21:10 - another clive james quote

In response to a quote on Greg's blog, I remembered another apt Clive James quote: "Like most people who organise their life badly, I just love giving advice."

wed, 22 mar 2006 11:25 - update on ben project

I feel that Ben and I finally have an effective, working 'spinner' prototype. Last week I was playing around trying to get the spinners looking more like propellers that spin around a single central point. I had my trombone with me and was making changes to the mass of the spinners and the length and tension of the springs that join them to the central point. The breakthrough came when I made the spinners much lighter and set the length of the springs very close to zero. The effect of this was that now the masses were spinning really fast and in a dense little ball which pulsed and glowed as they span. To me it looked a little like a mad little atom with protons and neutrons spinning around it on the edge of disintegration.

Exerting force on the spinning protons/neutrons by playing notes causes the particle associated with that note to spin out in a bigger orbit. Playing lots of different notes causes the whole structure of the atom to go nuts. This was fun to play with and to me appears to have a lot of possibilities. Ben seemed to like it when I demo'd it via iChat videoconference and he has a copy to play with now.

I've got a good feeling about this movement now. Conceptually it ties in nicely with the first movement, but in effect is quite different. The first one is slow and dreamy whereas this one seems on the edge of chaos, with much more rhythmic audio. I'm looking forward to seeing what Ben can do with it.

In terms of the creative process, it seems that Ben is now working to write a piece to go with the new spinners. That is, the music he wrote for the 'imagined' version of the software is not really suitable for the 'realised' version and so it (the music) will need significant revision. I think that the software development process as it is currently is far slower than Ben's compositional process. In other words, he can write good music much more quickly than I can write good software. For the first movement (Spheres) the music and software seemed to co-evolve more easily than they have (so far at least) for this movement. If we weren't living in different cities perhaps this would not be such an issue?

Another note: it helps when I have the trombone with me when I'm coding. Using audio files to test the software just isn't the same as playing live.

tue, 21 mar 2006 17:40 - shameless self-promotion

I spent the last day or so updating my crusty web page. It now has an up-to-date description of what I think I'm doing as well as a couple of videos of recentish work. It still needs a few photos I think. Maybe I might get around to putting some there soon.

wed, 15 mar 2006 10:30 - thummer

I notice that there is a PhD scholarship on offer at UWS to develop a taxonomy of interfaces for computer music, and that the industry partner for the ARC grant that funds this scholarship is thumtronics, a company that has developed what appears to essentially be a new midi controller. I have to say that at first glance the demos don't really excite me that much, but there are some interesting possibilties there. It's good to see an Australian company doing some innovative work in the area.

thu, 09 mar 2006 14:20 - ben reflection, etc

First up- before I forget, I notice that the recent Computer Music Journal has an article by Cyrille Henry, who wrote the physical modeling externals for pd. This is a reminder to access the artcle as soon as Project Muse makes the new issue available. (All the other journal databases seem to have a 12 month delay on this journal.)

Next, some updates. Firstly, Roger Malina is visiting CCS at the moment. His presentation outlined a case for 'deep' interaction between artists and scientists which provides a nice justification for the work Ben and I are doing- although I'm not sure I'm really in the role of 'scientist' in our case.

Also, last weekend was spent up in Brisbane - part ACID orientation, part Ben project work. Was a productive couple of days, largely trying out the prototype spinners (which we both agree are still pretty lame) as well as recording more versions of the spheres piece with (slightly) higher quality audio and generally talking over ideas. A big improvement in the spinners came when I changed the spinners so that they all rotated around the same central point. The picture below doesn't really show this too clearly, but the idea is that there will be a maelstrom of paper (or some other object) spinning around this central point responding to note articulations.

spinners prototype screenshot: now spinning around a single center point...

Ben has taken this a step further in the past few days, talking about the spinners working more like propellers. See this entry in his blog for details...

tue, 28 feb 2006 13:45 - vocal analysis

Following up on my ideas about analysing the more vocal qualities of musical articulation, I found Youngmoo Edmund Kim's PhD thesis on Singing Voice Analysis/Synthesis. Chapter 3 seems most relevant..

Later: Keith Martin's PhD thesis- Sound-Source Recognition: A Theory and Computational Model is referenced in the above and could also prove useful.

thu, 23 feb 2006 12:40 - some notes on spinner tweaking

While fiddling around with the spinners attack detection I changed some settings. Originally it was set to record 100ms of the attack of the note, but this wasn't really enough to hear the quality of attack clearly. 200ms seems to work better as you actually hear enough of the pitched note after the attack to get a proper feel for the overall articulation.

Another interesting point I want to check up on: for most of the notes I play, the sample graph shows more activity below the 0 line than above. I wonder what this means..?

wed, 22 feb 2006 11:30 - sibilance detection

I've been thinking about how to make the visuals in the Ben piece respond in more interesting ways to different styles of articulation (attack). At the moment the louder the beginning of the note, the faster the objects on screen respond, but I think this is too boring. I've read a few papers about onset detection, but these mostly talk about how to identify that an attack has occurred and don't go into how to identify its qualities. I scanned Robert Rowe's Machine Musicianship and found mention of a piece by Zack Settel - Punjar - which detected sibilant sounds in a singer's voice. I think that detecting the amount of 'noise' at the start of notes could be provide for more complex and interesting musical interactions.

sat, 18 feb 2006 10:30 - AECT handbook

After loooking through some old emails that I had flagged 'to do' and then let pile up, I found a link that Sam F emailed to the AECT Handbook. I don't know (yet) what AECT actually is, but thought I'd better blog it before I forgot because it looks like it gives a great overview of a whole range of philosophical perspectives on education and research, including chapters on User-Design Research and Visual Representations.

Follow up: AECT = Association for Educational Communications and Technology.

thu, 16 feb 2006 16:15 - lindberg and charlie

I got the trombone out from the filing cabinet for the first time in ages yesterday. I'd forgotten how enjoyable it could be to just play. Actually I think it's more of a physical experience than a musical one. Reminds me of the relationship between the act of playing and the musical concept. Anyway, while I was chatting with Ben this afternoon he mentioned that Christian Lindberg has an ongoing web documentary about the process of composing a concerto for Charlie Vernon and the Chicago Symphony. It's amazing to see Christian in action- I can't believe how much energy he has..

wed, 15 feb 2006 10:35 - steim

After browsing the Leonardo Abstracts Service, I found my way to Sergi Jorda Puig's PhD thesis: Digital Lutherie: Crafting musical computers for new musics' performance and improvisation. It appears to give a very comprehensive overview of the history of the development of digital instruments. I've been thinking lately about how the direction that Ben and I seem to be taking is to develop a kind of 'augmented instrument' in which the instrumental extensions are controlled by the sound as opposed to extra pieces of equipment attached to the instrument itself. The extensions can magnify musical gestures which can aid perception or provide new avenues for exploration. I haven't yet found any other examples of this approach, although I'd be surprised if there weren't any..

Oh, and the thesis made mention of the studio for electro-instrumental music in Amsterdam.

sat, 11 feb 2006 14:35 - george lewis

I'd read about George Lewis before, but hadn't blogged it. George is a trombonist who developed software which 'listened' to his playing and responded in what it decided were appropriate ways. I haven't been able to track down any audio on the net yet though- at least not one that works from within UTS.

I noticed today that Tristan Jehan, who has written some nifty audio analysis externals for max/msp, has his PhD thesis online. Probably worth checking out.

thu, 09 feb 2006 10:55 - alma

The ALMA project is working on sound synthesis using physical modelling. The basic idea is that a computer model is built which models all of the physical characteristics of a real instrument. So, if the model's 'strings' are plucked for example, they vibrate and the body of the instrument resonates, creating a very natural sounding effect. Obviously, the more comprehensive the model the more lifelike the results- and the higher the demands on the computer.

The webpage has a nice overview of approaches to sound synthesis.

...and later: stumbled onto Visual Music: Synaesthesia in Art and Music Since 1900, a book to accompany an exhibition at the Museum of Contemporary Art in LA. Unfortunately, not in the library..

...still later: Very useful bit of extra documentation for pd, describing how to create objects, subpatches, etc dynamically can be found here.

wed, 08 feb 2006 17:15 - the music animation machine

A journal article I was reading today (to be blogged later) mentioned the music animation machine- software which notates a piece of music in such a way that the individual musical lines become more visible than with conventional notation. While the software was clearly written some time ago (1985) and the visualisations are not real-time, it's quite interesting to check out. The historical time-line is worth a look too.

tue, 07 feb 2006 17:20 - pitch recognition/onset detection

The aubio project is developing real-time pitch-recognition, on/offset detection, etc. Website has some papers describing their approach. A pd plugin is apparently on the way.

sun, 05 feb 2006 18:45 - more thoughts on the Ben project...

Just following up on some further comments on Ben's blog. Ben observes that the collaboration has shifted from what we originally had in mind. What I proposed first up was that I play the role of 'technologist', giving Ben full artistic control. I think I had several reasons for proposing this approach. Basically, I was probably thinking that the 'requirements' for the music-learning toys (a significant research outcome) should not be 'contaminated' by my preconceived notions. Also, I felt that setting things up this way would help force me to wrestle with the software so that it became a more pure expression of an artistic vision rather than simply what was technically comfortable.

As it's turned out, I've been having a greater influence on the artistic side than either of us expected. I think there are several reasons, but the main one is simply because I have a greater awareness of what the technology is capable of, what has been done before (in this limited sphere - pardon the pun - anyway) and what is probably worth trying. Thinking about this as I write, it occurs to me that our approach is kind of like a very slow-motion improvisation. It's like we're kind of trading fours or something, in that Ben will play what he's been working on, I'll show some of the computer stuff I've been doing and we'll see how it can work. So we start at the 'playing bad jazz' phase, where the music and visuals are rough and don't really mesh together and gradually tweak everything until we start getting somewhere. Who's in charge, for me anyway, is a non-issue - but interesting to think about.

I can understand Ben's concern (based on past experiences) about the visuals dominating the music. One thing that I like about the little we've done so far is that the computer parts are really about building extensions to the solo instrument. That is, the computer generated visuals/aurals are controlled by the trombone and the music that Ben writes. They are not accompaniments to his music and neither is his music a soundtrack for a computer generated multimedia work. Maybe you could say that Ben is composing music for 'augmented trombone'?

thu, 02 feb 2006 09:45 - papers on semiotics/music psychology

Bill Thomson is giving a seminar: "Affective Channels: Music, Speech Prosody, and Facial Expression" on Friday as part of the AMPS seminar series. His webpage contains some interesting papers in the area of music perception.

...and later that day...found/refound the fftease~ objects for pd, which may have some interesting possibilities for the Ben project.

wed, 01 feb 2006 14:55 - music projects

Interesting page: here. Will check out fully later...

sat, 28 jan 2006 14:55 - reflection on reflections

Ben made some interesting comments on his blog, reflecting on the relationship between the co-evolving musical/visual compositions (spheres/spinners). Something I'm conscious of is the need for the 'virtual' part of each movement to have a distinct character yet to still belong with the other movements somehow. I think a strength of the collaboration so far has been a willingness on our parts to let the musical and computational compositions go their own way to a large extent and then bring them together and see what the interaction brings about. This also implies a willingness to make changes in response to the successful/unsuccessful elements of the interaction. Ben mentioned that the stuff we've produced so far has a 'calmness and beauty' - which I like - but in order to sustain interest over several movements a greater range is needed. This is a challenge to me to develop my range of expressive abilities on the computer.

A sidenote: A key characteristic of the computer stuff seems to be that relatively small changes to the output can have a huge impact on the character of the piece. For example, switching the shape of the spinners from spheres to triangles, to squares, etc changed the style quite radically. Perhaps I'm wrong, but I think this kind of effect is not so prominent in the composition of music. Perhaps it has parallels in orchestration??

mon, 23 jan 2006 11:55 - reflection on the weekend

I thought that the blog was a logical place for a reflection on how Ben's visit on the weekend went. The plan was to update each other on the progress of composition/programming and to develop the 'Spinners' concept further.

On arrival late Saturday morning I set up the extension/revision of the Spheres software (with the Spheres arranged in a circle, forces pushing outwards) and Ben played around with it, trying out bits and pieces of the Spinner composition with it too. I was pleasantly surprised at how quick and accurate the pitch recognition was and how this was reflected in the visualisation. The code tidying I'd worked on for the past couple of weeks really paid off in that respect.

From the point of view of developing toys for musical practice which enhance players understanding of their playing, there was a real 'breakthrough moment' while Ben was experimenting with the circle spheres. When slurring between two notes he noticed that in-between notes were being detected - because the spheres associated with these notes were flickering and moving at the change of note. When he played a truly smooth slur the spheres reflected this. He pointed out how sensitive and 'critical' the spheres were and how he never realised how untidy his slurs could be. It was cool that by playing with the toy he figured out how to play a noticeably smoother (to both of us) slur. I think that with additional tweaking of the various sensitivity settings, etc in the software this could actually work!

Ben spent some time playing with the circle spheres, trying different phrases/styles to see what what worked. During this time, I didn't actually do much to alter the setup because everything was responding ok and it would have seemed disruptive to change things too much while Ben was experimenting.

Eventually we moved on to the Spinners prototype which was in a far less advanced state. I ran the software and showed the spinners as they were - spheres orbiting around (invisible) fixed points. I think there were 3 spinners and low, medium and high frequencies respectively would cause force to be exerted on them. Obviously, the more force the faster the orbit. At this stage, the frequency of the input sound would determine the amount of force. A very simple mapping was used: low frequency = little force, high = large force. Unsurprisingly, this mapping was not very effective. It was too hard to see the relationship between input and visuals.

I made some changes so that volume instead of frequency controlled the quantity of force exerted on the spheres, and this helped. Ben also requested that the spinner that was being acted upon should flash or light up somehow so that it was more obvious what the relationship between musical action and computer reaction was. I hooked them up to the same sort of system that we used for the Spheres piece, so that the spinners glowed blue when they were being acted upon and gradually faded from view when they weren't.

It was interesting how critical the aesthetics of the visualisation were. For me it seemed that once I changed the spinners from spheres to squares, that we both started to see how the piece could work. As Ben was improvising and composing in front of the big screen, I changed the spinners from spheres to triangles, circles, etc. The most effective shape seemed to be squares because, as Ben said, they gave the appearance of papers blowing in a breeze (see screenshot below). At around this time I added more spinners (to a total of 6) to give the screen a busier appearance. We also discussed having a much larger number of smaller spinners (maybe even hundreds?) on screen, each responding to more specific frequencies. This was left for later work as it would have taken too long to do it on the spot. I also experimented in texturing the squares with video/pictures/words/letters, etc. We agreed that they seemed superfluous.

Spinners prototype after the weekend...

Ben felt that the program should provide audio feedback also. He mentioned that when he was playing with the circle spheres that the sounds produced in response to his playing gave him more to work with- provided a kind of platform from which he could launch ideas. I hacked together a quick output patch to give us a starting point. Conceptually, each spinner outputs a frequency within the band that causes it to move. I chose the frequencies arbitrarily for ease of coding reasons. Lowest frequency spinner outputs 60Hz, next 120Hz, 180Hz, etc... The faster each spinner spins, the louder it outputs its particular pitch. The effect is like a drone which changes timbre based on what Ben plays. This worked somewhat, but the drone was too boring and didn't seem to really change enough.

We talked about making use of sampling somehow, so that the piece would record snippets of Ben's playing and play them back. I implemented a recorder which could store short (1-2 second) snippets of live audio into 6 banks (ie. one per spinner), but then stalled when I was trying to think about how recording and playback would be triggered. (eg. audio beyond a certain volume threshold triggers playback, or maybe specific frequencies trigger playback?) We were getting tired at this stage and spent time on the way back home talking about ideas for the audio. The most promising idea is to have the spinners act a little like ratchets- each with their own special timbre and perhaps physical characteristics (ie. hard to spin, easy to spin, etc.). These would act like percussion instruments activated by Ben's live sound. I like this idea! Next step is to try it out...

tue, 17 jan 2006 13:50 - ben project update and todos

I've managed to do a few things on the spheres project that have been bugging me for ages. Ben and I have spoken about arranging the spheres in a circle for a while as this seems a conceptually stronger way to present the twelve pitch classes. So I spent some time figuring out the best way to organise the masses/springs in pd in order to get this working. The solution I ended up with was to suspend the twelve masses in the middle of a long spring (like a string on a violin) which is fixed at both ends. One end of all the strings is fixed to a single mass in the far distance. The other end of each spring is fixed to one of twelve fixed masses (which are not shown on screen) arranged in a big circle. If the string is tensioned appropriately, the end result is a circle of floating spheres, each of which responds to a particular pitch class. When a note is played by the musician, the associated sphere is pushed outwards with a force proportional to the volume of the sounded note. The effects can be intriguing. What I like is that if you play a short, loud note it's like 'plucking' the string that the sphere is attached to. Because the sphere still 'stores' the frequency which caused it to move and plays it back as it moves, careful note selection means you can 'play' this virtual stringed instrument. I think this has some cool possibilities...

I also tidied up a lot of the messy code from the original version. For example:

• Pitch analysis is now rounding properly. Previously the recognised pitch from fiddle~ was simply truncated to an integer! It now rounds up/down appropriately.
• The spheres colour/glow is now determined by the force being exerted on the sphere. The more force, the more it glows. Previously the volume of the pitch and component frequencies controlled this. It still does actually, but now it is via the force-manager component. Also, the glow gradually fades when the force is no longer acting. Previously, the glow would continue until fiddle~ detected the same pitch at a lower volume in the input again.
• Forces now only push outwards. In the early versions forces could push inwards too.
• The mass of '0-mass' is now set correctly. There seems to be a bug in the Gem counter object- don't use this in future.
• Removed the superfluous stereo pitch analysis which we tried when we had the idea of 'pushing' spheres around based on the direction that the musician was pointing the instrument. Performance is much better in mono!

tue, 10 jan 2006 14:00 - now blogging with rss?

I've been resisting the CCS blog revolution for a while because I've had my own lo-tech blog which didn't hook in to the collective voice - until now.. Since I've had a little time on my hands I wrote a little program which converted my simple blog into the required format. If this can be read then it works and now I need to find some other procrastination excuse..

wed, 04 Jan 2006 16:10 - installing pd/gem on fedora

Setting up pd/gem, etc on the work machine now running fedora core 4 is taking a little fiddling. Finally got the latest version of GEM from CVS to compile. Had to remove some source files as discussed in this post on the pd list and in addition remove the "glUseProgramObjectARB.*" files from the src/openGL directory. This compiled and seems to be working fine so far. Next step is to install pmpd...

..and pmpd is working now. The final (I thought minor) dependency for the spheres piece was maxlib (which is only used for scaling, but is very handy). It took a while, but after downloading the latest cvs version of the pd 'externals' from sourceforge, I eventually got it to compile. The trick was to copy the 'include' directory from ../externals/build/ to ../externals/maxlib and then do the 'make pd_linux' command, which seemed to work ok.

fri, 30 Dec 2005 10:30 - where did you go?

Well, the end of the year was a real mixed bag. Too much happening to really make any progress on PhD, but some fun stuff such as the trip to CELDA in Portugal and a trip down to Melbourne for Xmas. Anyway, this entry is actually a reminder to read the rest of the winter Computer Music Journal (available on project Muse) which has a lot of articles about visual music. I briefly read through the Dannenburg article which had some potentially useful info about his approach to composing interactive visual/musical works. Read the rest!!

mon, 21 Nov 2005 09:45 - readings...

While glancing at some research papers on my way to start more marking (groan), I stumbled on some interesting sounding readings in the area of development of expertise. First up, this site, an outline for a UTS MEd subject, Skill Learning and the Development of Expertise, has a useful reading list. It includes the book The Road to Excellence: The Acquisition of Expert Performance in the Arts and Sciences, Sports, and Games edited by K. Anders Ericsson.

One day I might get some time to have a look at these?

Also, Ernest has requested that publications for 2005 be blogged. Here they are:

• Johnston, A., Amitani, S. and Edmonds, E. 2005, "Amplifying Reflective Thinking in Musical Performance", Creativity and Cognition 2005, London, ed. Linda Candy, ACM Press, New York, pp. 166-175
• Weakley, A., Johnston, A., Edmonds, E., Turner, G. 2005, "Creative Collaboration: Communication, Translation and Generation in the Development of a Computer-Based Artwork", 11th International Conference on Human-Computer Interaction 2005, ed. G. Salvendy, Lawrence Erlbaum Assoc Inc., Las Vegas, Nevada USA. CD ROM.
• Johnston, A., Marks, B. and Edmonds, E. 2005, "'Spheres of Influence' - an Interactive Musical Work", Interactive Entertainment (IE2005), Sydney, Australia (in press)
• Johnston, A., Marks, B. and Edmonds, E. 2005, "An Artistic Approach to Designing Visualisations to Aid Instrumental Music Learning", Cognition and Exploratory Learning in Digital Age (CELDA 2005), Porto, Portugal (in press)

saturday october 29, 2005 - ben notes

Change to spinner movement plan. The pitch now selects which spinner will spin, but does not determine speed. Speed is instead determined by volume. Pitch is divided into three zones - low, med, high - which map to the three spinners. Current volume then exerts force to speed up spinner.

Try- faster spin means greater mass (but subtle, not huge effect...)

Also, within the screen spinners can move around. Perhaps affect each other...a la hot gas..? Couple with colour changes- more intense, hotter look.

tuesday october 25, 2005 - fugueart

Found fugueart, an arts-science collaboration which the creators call an 'audiovisualisation'. The basic idea is to produce an artwork which facilitates an understanding of the workings of the immune system. Could be a helpful reference for my work..

tuesday october 11, 2005 - slack blogging: David Rosenboom

Been super slack in the blog department lately. Probably because everyone in CCS is now required to blog I've lost the urge. Anyway, was revisiting George Khut's webpage while preparing for my upcoming doctoral assessment and followed a link to David Rosenboom's home page, an artist with a long and illustrious history in art-music.

friday september 9, 2005 - 'musical excellence'

Been looking at Musical Excellence edited by Aaron Williamon. In particular today I read the chapter by Harald Jorgensen: "Strategies for Individual Practice". It's a useful chapter especially for getting an overview of research in this area. Anyway, an area that is mentioned is reflection on practice. That is, thinking about how successful/otherwise a session was, what went right, wrong, why, etc. This led me to think about how tools for real-time feedback should be able to be used in a reflective way as well as while actually playing. This means probably providing some facilities for recording/playing back in the visualisation framework. This would give the additional benefit that the user could tweak settings to view the performance in different ways as it is being played back.

This ties in nicely with Shigeki's mirroring drum idea too. The recording feature may be effective in live performances too, a la mirroring drum, but I'm not sure how yet. Hmm.. how about this: musician plays and visualisation works in real-time as well as recording musician's performance. Then, musician triggers playback of the recorded stuff and uses additional musical input to control the visualisation of the playback. Is this too recursive?? Maybe it's just too clever by half, but worth thinking about..

thursday september 1, 2005 - leonardo article

Was browsing the recent Leonardo and found this great article which almost completely explains my position on developing s/w to support musicians. Describes interdisciplinary approach using a design methodology. Interesting that the medical applications of the visualisation framework they discuss grew out of their work as artists. That is, the artwork led technology development, rather than following or using the technology developed for medical purposes. Reference: Bermdez, J., Agutter, J., Foresti, S., Westenskow, D., Syroid, N., Drews, F. and Tashjian, E. 2005, "Between Art, Science and Technology: Data Representation Architecture", Leonardo, vol. 38, no. 4, pp. 280-285.

tuesday august 30, 2005 - natureFX

Been working on second Balls movement: spinner. Got some basic stuff going, but was looking in my book on computer animation and effects and found mention of NatureFX, a program which can generate naturalistic sea-surface animations. Could be useful. Dynamic Realities is the company that produces it, along with a number of other plug ins for Lightwave that simulate natural effects.

saturday august 27, 2005 - balls in breeze II

Brainstorming for second movement of the balls piece. Basic idea: spinning tops (maybe 3? more??) louder notes speed up spinning (or maybe vice versa?). Spinning rate is the frequency of input note(s) (or component)? Frequency of input note sets maximum spin speed. Volume sets rate of acceleration. ie. Loud note at 100Hz means spinning increases quickly to 100Hz. Softer note still causes spin to increase to 100Hz but more slowly. Tops are always in a limited frequency range, but C's will always result in the top spinning at a C frequency, but perhaps at a lower or higher octave.

Idea for presentation of balls mvt I: microphones are "feelers" for computer. ie. make the cables clearly come out from the screen and lead to the mics.

Another idea for mvt II: Three bands of pitch (hi med low), one for each spinning top.

Tops spin on something solid/visible.

wednesday august 24, 2005 - analysing interactions

Been too long between entries. Will catch up soon- after Ben's visit this weekend. Anyway, Shigeki posted the link to Interaction Analysis: Foundations and Practice. Looks like a potentially useful paper on techniques for analysing interactions in situ.

Useful GEM tip from pd mailing list: "note: the height of the gemwindow will always be 4 units (the width will change with the aspect-ratio)". Some more snippets from emails to the list:

You said something that I find interesting as I'm trying to learn Gem, "The height of the GemWin will always be 4 units?" How's that?

oh, i have to apologize and become more accurate: an object with height 4 at position 0/0/0 with the camera at the default-position (0/0/-4) and the default perspective-setting will touch the upper and the lower border of the window exactly.

Where was I supposed to find about that? I have read a few >> documentations and none of them specified this... Do you recommend one in particular?

i guess this is no-where specified, what i have told as a fact has been gathered empiristically.

the important thing to remember is: in Gem you are viewing with a camera onto a 3D-scene. under certain circumstances (relative position camera vs. object; camera settings) the abstract units can be mapped to pixels rather simply but these are "special cases".

Also mentioned was this page containing pd presentations from the workshop in Graz.

saturday august 13, 2005 - pcp and the arts / clatterbox

I followed up on an email sent by Joern Sheer to the personal construct psychology (PCP) discussion list about the use of PCP in the arts. apparently there is a book due out soon and they are interested in hearing about people's experience in this area. Joern has set up a web page. There is a password-protected section (user name: allarts, password: eurtsnoc) with contact details of other researchers, etc.

Another useful site I'd been meaning to blog for a while: clatterbox. It's a website "profiling Australian experimental instruments".

monday august 8, 2005 - new online journal

Rod posted a link to a new journal in the area of music learning, Reviews of Research in Human Learning and Music. First issue is due in (US) summer 2005.

friday august 5, 2005 - simon holland thesis

While reviewing papers for IE2005, I found Simon Holland's PhD thesis on Artificial Intelligence, Education and Music. He seems to have looked at the potential of AI to support the development of skills in composers rather than musicians, but the thesis appears well written and shares some obvious common ground with my work. A nice reference.

wednesday august 3, 2005 - alan davis

...is visiting UTS requirements engineering group. Noticed he is presenting a seminar next week on "Rise and fall of a software startup" which sounds like it might be interesting. Also he has books out: Great Software Debates and Just enough requirements management : where software development meets marketing, which both sound very relevant for the XP class. A good idea might be to photocopy a chapter from the first book which Alan says is about whether to use agile methods or not for a project (chapter 4).

sunday july 31, 2005 - in the chair

Presented at the MIFB yesterday. I felt the presentation went well and there is certainly some interest in the teaching community. Howard Parkinson (bass trom, ASO) was there and mentioned an amazing project that the ASO is working on: In the Chair. This is basically a kind of "flight simulator" for musicians. Using the software they can virtually "sit-in" with the orchestra and play along. The software gives feedback on quality of sound, intonation, style, etc. It's been developed by Digital Monkey. Don't know how effective this is and just what parameters are being analysed and in what way, but the basic idea sounds brilliant.

thursday july 28, 2005 - garth paine

Well once again while working on MIFB presentation, I found an article on Realtime about Garth Paine who used to be a flautist with TSO and is now doing really cool-sounding stuff in the interactive music area. Could be a good co-supervisor?

wednesday july 27, 2005 - george khut, mindpeak

While working on Melbourne International Festival of Brass presentation, I checked out George Khut's work again, especially Drawing Breath. While there I noticed a link to Mindpeak, a company that makes various devices for making physiological measurements with computers. I should talk to George about the chest bands he uses to measure breathing activity.

tuesday july 26, 2005 - worm, etc

Working on my presentation for the Melbourne International Festival of Brass and was trying to find the videos of the worm visualisation. They are here.

friday july 22, 2005 - mixer

Had a quick look at mini mixing desks today. Behringer Eurorack UB1002 looks suitable. Has 'fx to control room' switch which 802 and 502 don't have. Also, you can assign tape input to either main mix output or control room output, so could connect computer output to tape input and assign to control room out and mix two mics to main output and send to computer input and avoid the problem of amplifying live instrument. RRP according to Behringer website is $129.00.

wednesday july 20, 2005 - nice journal article

Got a contents alert from somewhere for the Music Education Research journal, which had an interesting artcle: Real-time feedback in the singing studio: an innovatory action-research project using new voice technology by Graham Welch, David Howard, Evangelos Himonides and Jude Brereton. I like their approach - using action research to justify a participatory design approach to software development. The paper includes a comprehensive literature review. One book which looks interesting is Musical Excellence edited by Aaron Williamon. Library has a copy on loan at the moment. Must check it out.

tuesday july 19, 2005 - update & physical modeling based synthesis

Well it's been a ridiculously long time between entries. Lots of stuff been happening. Firstly, going to the Melbourne International Festival of Brass soon to present the Spheres piece with Ben and to talk at the PD day on technology in teaching/practice. Should be fun to catch up with all the brass people again.

Submitted papers to Cognition and Exploratory Learning in the Digital Age (CELDA), Interactive Entertainment and ACIS in the last few weeks, which partially explains the reason for the lack of activity in the blog department. Also, as PEP has now officially ended, I've been getting distracted by the usual work stuff.

Anyway, while browsing through the backlog of PD-List messages I found some discussion on Percolate, a set of externals for Max/PD which use physical modelling as the basis for sound synthesis. These could fit in quite well with the spheres ideas...

monday june 27, 2005 - spheres piece / messa di voce

After we demo'd the spheres piece, Lizzie mentioned a project called Messa di Voce which has similar characteristics to ours. When I looked it up, I realised I'd seen a cut-down version at the Ars Electronic centre in Linz. The "full" performance version seems much more interesting from the videos on the web-site. This should be cited!

Lizzie also mentioned Jonathan Impett, who developed the metatrumpet. It's hard to get a clear idea of just what this instrument is about from the web site. While googling around for more info, found this paper, Physical Interfaces in the Electronic Arts by Bert Bongers. Great name for a music researcher anyway. On the reading pile...

And for contrast, here is a rough and ready video of Ben playing the spheres piece (3 minutes, 11mb Quicktime), recorded on June 23 at CCS.

wednesday june 22, 2005 - ben todos

Things to work on by the end of this week:-

• Draft report using application criteria
• Video performance
• Draft score
• Copy of program for ben
• Possible uses in teaching brainstorm

monday june 20, 2005 - ben todos

Some stuff to work on:

• Movie/volume: make volume response more linear (convert from db?)
• Balls: can vibrato move ball up and down? Pitch still selects ball, but vibrato on that pitch moves ball vertically...
• Make a link between balls and audio playback. ie. Ball movement controls volume of buffer playback. Each ball is linked to 1 buffer.
• Make the balls make a "clinking" sound or similar when they hit each other.

saturday june 18, 2005 - s2s2

Received an interesting call for participation for the Sound to Sense, Sense to Sound (S2S2) workshop in Genoa. It looks like you have to apply to participate, but I doubt it would be DEST fundable.. Check this out further.

wednesday june 15, 2005 - pmpd stuff / michael casey: musical feature analysis

Well things are quiet at uni today, so I managed to actually get a little work done. Been fiddling with pmpd more and am starting to get the hang of it slowly. Thoughts of the day:

• I'm trying to keep all forces, masses, link lengths, etc between 0 and 1. This is an attempt to stop the complexity of interacting masses and forces and so on causing weird/unpredictable behaviour.
• The 09_tutorial2D.pd in the pmpd documentation is a good place to get a feel for pmpd.
• Use the Xmin, Ymin (and Zmin) mass parameters to set boundaries for masses on the screen. Because GEM coordinates seem to be between -4 and +4 I've been keeping the pmpd coordinates in the same range.
• The difference between 'normal' damping and 'velocity damping' for links in effect seems to be that normal damping will cause oscillation and velocity damping won't. If a fixed mass and a free mass are linked together with a link which has only velocity damping and are "stretched apart" then the free mass won't spring back past the fixed one. With normal damping it would. I think..
• Remember to 'bang' the links first, then the masses during simulation.
• To reset links send: reset, resetL and resetF in that order. (At least that's how the tute does it...) To reset mass send reset.

Ok, and on another matter, I was looking through my notes on the meeting I had with Geraint Wiggins and noticed that I had a reminder to check out Michael Casey who is doing research into musical feature analysis. Lots of interesting-sounding projects are listed from his home page in the area of attempting to identify important musical characteristics, mostly at a higher (structural) level mostly it seems.

tuesday june 14, 2005 - book: music matters

Music Matters by David Elliott seems to be cited fairly frequently. The library has a copy which is on loan. I put a request for it to be delivered to city campus today..

Another potential name for the discussion list: Musical Creativity and Technology.

wednesday june 8, 2005 - (yet another) journal

Found the Music Perception journal. Looks pretty promising.

monday june 6, 2005 - rep grid supplied constructs

More thoughts on supplied constructs for rep grid interviews when eliciting elements with "Things you notice about your playing". Current list is: "easy to notice -- hard to notice", "need reminding to notice -- don't need reminding", "important -- unimportant", "students generally need help to notice -- students generally don't need help to notice", "have always noticed -- learned to notice".

friday june 3, 2005 - jWebUnit

Stumbled on a link to a useful-sounding tool for XP: jWebUnit...

Reminder: Look through the proceedings of ICMPC8! (CD from Geraint Wiggins)

Even though I should be writing the paper for IADIS/CELDA I found musicpsychology.net which looks like a clearing house for papers about aspects of music. Not that many papers there, but could keep an eye on this..

thursday june 2, 2005 - digital creativity journal

Rodney posted a CFP to Digital Creativity, which looks an interesting journal. They are especially interested in experience reports, etc. from artist/technologist collaborations. Due date is October 1.

Also, found proceedings for the Sound and Music Computing 2004 conference. Some interesting sounding papers there...

wednesday june 1, 2005 - more on rep grid interview

Interviewed Matt Walmsley last night and am starting to get a clearer idea about how to go with the rep. grid process. I think the next time I should elicit elements with: Things you notice about your playing.... To avoid software complications I think I will write the elements on index cards after entering them on the computer. I think also, I will make it clear at the outset that element elicitation will take around 5 minutes and treat it more like a brain-storming sesssion. The idea is that in depth thinking about practice will come when we get to construct elicitation. In most of the interviews so far I think we have spent a little too long on getting the elements out and then been tired/rushed when getting contructs sorted.

The advantages of the 'things you notice' approach are that:

• It's easier for the subjects. They often mention how confusing it is to "think about things you think about"...
• It provides a more direct link with software analysis. If we can identify the things they notice/want to notice we can build software to amplify the perception of these things.

Another good idea which I got from The Easy Guide to Repertory Grids by Devi Jankowicz, is to rate the elements in the triad used for construct elicitation first, before going through all the other elements. Repgrid automatically assigns values of 1 and 5 to the similar/different elements and I've found that often the interviewee doesn't want them assigned that way.

thursday may 26, 2005 - rep grid summary

Just thought I'd jot down my current repertory grid interview method. Elements are elicited with, "things you think about while practicing..." I'm toying with the idea of making this, "things you notice about your playing while practicing", but I'm not sure about this.

Constructs are elicited using the triadic method mostly, but at times I will show the entire list of elements and ask them to identify two which are different and explain why. In addition to the elicited constructs I'm now providing some (after elicitation): "important -- unimportant", "think frequently -- think infrequently" and "students think frequently -- students think infrequently". The aim is for these constructs to provide some rating on the perceived value of the elements and differences between players perception of themselves vs. 'typical' students. Other constructs I'm considering include: "need prompting to think about -- don't need prompting" and "learned -- came naturally".

Also, after talking with Ernest about PhD scholarships I had a look at the graduate school website. Found details of Australian Postgraduate Award (APA) scholarships there. I think this is what Jenny E has mentioned previously and are for around $19,000 tax free. In addition, Ernest mentioned ACID as a potential source of scholarship funds..

wednesday may 25, 2005 - pure data goodness

Last couple of days I've been back working on Ben project stuff, in particular the "sheres of influence" or whatever. It's now possible to texture pictures on each sphere and I've also hooked it up so that individual partials of the sound can exert force the relevant spheres also. eg. the main pitch exerts force in the X direction (proportional to its volume), secondary pitch in Y, etc. This may be effective artistically but I don't know if it will be of much benefit for musical feedback. Anyway, will need to experiment to find out.

Oh yeah, volume can also affect the transparency of the pictures on each sphere now.

monday may 23, 2005 - HCSNet

Sam passed on a link to HCSNet, a research group looking into "Human Communication Science". Don't know exactly what that is, but apparently they are a good potential source of funding. Poking around their website I found they have grants for workshops, and it occurred to me that this could be a worthwhile project. Would be good to get a group of people together who are interested in this area... This is on the list of things to raise with Ernest on Thursday.

thursday may 19, 2005 - principle component analysis

While browsing the idiogrid web page, found these videos on principal component analysis. A good overview of how to interpret PCAs.

wednesday may 18, 2005 - repertory grid, etc

First up, the web page showing the acoustics and audio seminar schedule is here.

This morning I spoke with Lisa Fry, who is doing a masters with Dianna Kenny on aesthma patients adherance to medications. Dianna has pointed her towards repertory grid and she is trying to get her head around it.

While on rep. grid, last night I interviewed Andy Evans. This interview was harder than the one with Ros. The elicitation of elements was no problem and Andrew quickly identified 14 "things he thought about while practising". It was much more difficult to get the constructs though. Quite likely this was because Andrew is not used to thinking about these issues in terms of construct poles. Observations:

• The interview took too long. Andrew was getting mentally pretty tired by the end (as was I).
• Andrew grouped some elements together as "similar" when I would have said they were "related" instead. eg. When presented with 3 elements he selected say, "breathing" and "legato playing" as being "similar". When asked how, he said that when he was breathing well his legato playing was good. Obviously this is a perfectly valid observation on his part, but in terms of rep. grid it's hard to get poles out of that statement. Should I have suggested the construct, "related to smooth playing" -- "not related to smooth playing" or "improve smoothness" -- "don't improve smoothness" perhaps? This is an important aspect of my interviewing technique I think. I need to get quicker at picking up on this and get some strategies worked out.
• Andrew was interested and intrigued by the methodology and the analysis tools.
• It could be worth considering using index cards and entering constructs on a piece of paper by hand rather than the software. I think this may help remove distractions, especially because the repgrid scripts are pretty confusing. Maybe a hybrid approach of entering elements/constructs and ratings into the software but also writing the elements on index cards could be good.

friday may 13, 2005 - trip reflection III

Continuing from yesterday with my notes from my visit to Plymouth, the final person I met was Marcel Gimenes (marcelo DOT gimenes -AT- plymouth DOT ac DOT uk) whose thesis topic was Musical ontogenesis in artificial society. Ontogenesis is a term coined by Marcel and Eduardo to mean a sequence of events involved in the development of an individual style. So in other words Marcel is looking at the development of musical style using AI/A-Life techniques.

Anyway, Marcel's work was really interesting. He's trying to model the development of musical style. He talked about 'memes' (Dawkins) as being a social equivalent of genes (i think). I would like to read some of his papers (although he's in the early stages too) because the ideas are interesting. As with a lot of the AI research though, I'm not sure I quite understand how the methodology works.

So, some brainstoming type ideas for collaborations with Eduardo's team follow:

• Use of AI techniques to identify musical elements -> visualisations for musicians.
• Qijun's project on shaping individual notes: possible use in transforming performer's input??
• Andrew's use of eeg? Could help identify 'states of mind' of musicians? (This is a PhD in itself..)
• Eduardo's project? Could 'identify' the emotional effects of performances -> highlight via visualisations?

This afternoon, I borrowed a demo DVD of George Poonkin Khut's piece, 'drawing breath v.3' from Lizzie. The DVD was actually put together as part of an Australia Council grant. The basic idea is to provide the participant with visualisations of their breathing/heart beat. The DVD seemed to highlight the breathing aspect more and to my mind this was the most effective part, and also potentially the most interesting from the perspective of trying to find useful visualisations for musicians (particularly wind players). The person credited with the 'realtime data analysis and visualisations' was John Tonkin, who has done quite a bit of cool stuff. Particularly like the morphing bodies/faces stuff...

thursday may 12, 2005 - trip reflection II

Ok, gradually writing trip stuff down. Today I thought I'd edit/paste/update my notes from Eduardo's group at Plymouth.

I first chatted with Eduardo Coutinho who was actually on his way to a conference right after our chat, so I really didn't get a completely clear idea about what he was up to. Eduardo C is looking at modelling the emotional effects of music using intelligent agents/A-Life techniques. I had trouble understanding exactly how he was going about this, but the basic idea has some interesting possibilities for my work. Maybe the computer could predict the emotional effect of a performance? Also, he recommended a book by Damasio (a neuro-psychologist- emotions, etc) called "The feeling of what happens".

Andrew Brouse(andrew DOT brouse -AT- plymouth DOT ac DOT uk) is working using an Electroencephalograph (EEG) machine to control computer music directly with the brain. He has only recently started and is working on technical issues involved with hooking up the eeg via Open Sound Control (OSC) protocol. This basic approach can be used to send the data to any OSC compatible program.

He showed me a video of a performer who was 'playing' a piece (originally described by ??) where the performer was connected via EEG to various percussion instruments. When the performer generated 'alpha' waves - occurs when relaxed and eyes are closed (?) - the drums were triggered. The sound did seem to me to have a pulse, but not sure about this. Anyway, the intensity/density of the rhythm varied during the performance - I assume the performer was directed to think different thoughts or something? When the performer opened his eyes the playback ceased. My description doesn't do this justice- the piece was quite effective.

Andrew mentioned he was planning to attend the Neurosciences in Music conference (Leipzig) in May.

Interesting book: The Conductor's Jacket by Teresa Marrin Nakra : Study of conductors physiology while conducting.

BTW, MatLab is the tool for matrix analysis- audio/image processing, etc. that Andrew and others are using. Can do real-time analysis too.

Prof Adolfo Maia Jr. is a visiting professor (adolfo-AT-nics DOT unicamp DOT br) who is a Mathematician: Cosmologist/Astrophysics interested in Granular synthesis; mixing markov chains and fuzzy sets. He was extremely good at explaining the concepts underlying his work, but it is nonetheless involves quite complex mathematics. Possible suggested readings included: Gabor: sound quanta and Roads: application of these ideas to music. A recent paper was Fuzzy granular synthesis of sounds with markov chanins by Eduardo Miranda and Adolfo Maia in the Acoustical Engineering Society Journal.

I didn't really understand exactly what Adolfo was doing, except that it was a kind of extension of granular synthesis using markov chains combined with fuzzy sets. The program (in matlab from memory) was called 'fuzkov' and when fed various parameters would process an incoming signal (sine wave when demo'd) and output the 'granularised' version. Adolfo was very good at explaining the mathematics of what was going on. From memory his description of fuzzy sets was that they were a bit like clouds. Clouds are made up of particles (of air/water I guess). Particles in the middle of the cloud are certainly part of that cloud (ie. part of the 'set'). When looking at particles at the edges though it becomes harder to tell, because the edges of clouds are 'fuzzy'. I think he said that in fuzzy sets there is a probability of belonging to a set which is assigned to each particle. I assume that particles in the middle have a very high probability of belonging and those at the edges have probability of belonging approaching zero.

This idea, combined with Markov chains, is used during the granular synthesis process. The actual output was hard to evaluate, as the sine wave was chopped up into fragments with large silences between them. I'd need to hear it applied to real sounds (and in real time). Processing in real time is currently a problem due to the large amount of maths involved, but there's no reason in principle why this can't be done.

Vadim Tikhanoff (a masters student) is working on a robot soccer controller for max/msp which translates robot sensor info, etc into midi messages. Vadim has written a program which runs in conjunction with the sony aibo robot dog and max/msp/jitter. Using this program, you can control all aspects of the dog's behaviour, get input from its video camera, etc. Basically the program converts all the sony network protocol events into midi events which can then be processed by any program (so this is not really tied to max/msp). I can't remember how he dealt with the video. I don't expect that he used midi for this..

Joao Pedro Martins (joao.martins-AT-plymouth DOT ac DOT uk) has an engineering background and is working on an AI approach to the emergence of rhythms in a 'society' of agents. The idea is that agents encounter each other and listen to each other's rhythms. Agents are attracted to and influenced by other agent's rhythms and the rules that govern which choices agents make are configurable in Joao's program. He mentioned that some research has shown that people have a 'natural tempo' towards which they tend to drift when tapping a given tempo. I think he is incorporating this idea into his simulation. Another interesting aspect of his model is that the agents have an 'arm' that they use to beat tempos, so there is some kind of physical model which affects their playback, sets limits, etc.

He ran some simulations for me so I could hear the agents beating their drums and gradually moving towards a stable pulse. Depending on the rules and starting conditions, the agents may take a very long time to move towards a stable pulse and even if they do settle on a steady rhythm it's possible they will diverge again.

The ultimate aim, I think, is to discover some aspects of human cognition/social interaction by creating a model which exhibits human-like behaviour and then looking at the programmed rules that lead to this behaviour.

Recommended reading: Listening - An introduction to the perception of auditory events by Stephen Handel.

Qijun Zhang (qijun.zhang-AT-plymouth DOT ac DOT uk). Qijun's work was really interesting, but in the very early stages. The software called superconductor allows very precise control of the playback of many aspects of music. For example, she demo'd a classical string piece and you could change the shape of each note, the attack, vibrato, etc. You could do this globally for *all* notes- it was not necessary to individually edit each note. Apparently a drawback of this software though is that it uses its own proprietary format which is kind of like midi but contains much more info. Qijun is working of implementing similar features but with standard midi files.

I'd really like to see these kind of features available for manipulating audio files rather than midi, but this is obviously a huge task. From the teaching perspective this would be pretty cool. Actually, thinking about this, I'm sure there are a lot of recording studio digital editing techniques that *could* be applied in real time already- vocoders, etc. This is something I really need to look at in more detail down the track.

Reading: Manfred Clynes - theory of music structure, shaping of notes.

wednesday may 11, 2005 - fledgling research group

Had another meeting of our mini research group this morning with Sam Ferguson and Shigeki and discussed the possibility of applying for an Australia Council grant (or other suitable grants). Sam mentioned that the Architectural and Audio Acoustics Research Group at USyd runs seminars every Friday, and Shigeki and I plan to attend. Sam is doing a pre-conference presentation of his InfoVis2005 paper on Friday week (May 20).

Some todos arising from the meeting:

• Write a short (1 page or so) outline of potential grant application for feedback at next meeting.
• Set up a mailing list. (emailed sbg about this)
• Send my blog link to the others.
• Check out pitaru.com. Done- cool 3d musical sculpture. Uses processing, a combination java API and programming environment for graphics/sound programming.
• Check out 'sounding out', an Australia Council thingy for supporting the development of new musical instruments?
• Check out DAFX conference.

While googling DAFX, found Gary Scavone's homepage containing a lot of papers on instrumental acoustic processing/simulations. Just something else to follow up...

...AND...via the PCP-List I found the Personal Construct Theory & Practice journal.

tuesday may 10, 2005 - experimenting with personal construct psychology

My interlibrary loan book, Experimenting with Personal Construct Psychology arrived yesterday and I read through the three most obviously relevant articles this afternoon. At first reading they seem to address higher-level concerns with PCP rather than discussing actual use of repertory grid techniques in investigating music, but need to re-read.

Also, on the chi-announcements list received call for papers for a SPECIAL ISSUE OF SIGNAL PROCESSING (ELSEVIER) ON MULTIMODAL HUMAN-COMPUTER INTERFACES, which lists musical interfaces as one of its interests. Should consider further. Submission guidelines are here.

monday may 9, 2005 - vise grant, australia council, ...

Wrote a two page application for a vise grant to try to get funding for a couple of Ben visits to put together an Australia Council application to develop a new work. Total of $1,720.00 applied for, which was for two return trips plus accommodation and meals. Also, Emma forwarded a call for papers for "Backing Our Creativity, a National Education & the Arts Symposium to be held in Melbourne 12-14 September 2005" which is apparently being run by the Australia Council.

While trying to find out more about this I found the Australian Society for Music Education, who are running a conference this year (and every year??). Sounds like it could be a useful group anyway..

friday may 6, 2005 - computational aesthetics

Came across the computational aesthetics conference today. Too late to submit anything and not properly peer reviewed, etc., but I like the sound of it.

Yesterday, did some work with pmpd to get the Ben project stuff underway again. The relationship between forces, masses and movement is quite complex. I need to figure this out. I know that pmpd only works in terms of 'units' rather than meters, kgs and so on, but I still need to get a grip on how to choose sensible figures to get the effect I want. May need to dig out the physics textbooks...

wednesday may 4, 2005 - sam ferguson meeting

Had very pleasant discussion with Sam this morning over coffee. I think our interests are definitely complimentary and I think it will be very worthwhile to discuss issues and perhaps work together on some projects. We teed up a meeting next Wednesday at 10.30 to include Shigeki and perhaps Sam's PhD supervisor. The plan is to meet regularly and pretty informally to keep up-to-date, discuss issues, perhaps present papers/works in progress, etc.

tuesday may 3, 2005 - sam ferguson

Made contact with Sam Ferguson, a PhD student at the School of Architecture, Design Science and Planning at Sydney Uni. Sounds like we share many similar ideas for using real-time visualisations for musicians. Meeting tomorrow morning for coffee.

While googling Sam and co-authors for the paper he sent me, I came across the Information Visualisation conference which is similar to the InfoVis conference I mentioned yesterday.

monday may 2, 2005 - new journal

While going through about a million CFPs for conferences on the CHI announcements list I found a new journal which may be worth keeping an eye on, although it does sound pretty techo- the ACM Transactions on Multimedia Computing, Communications and Applications.

Also, the ACM Multimedia 2005 conference looks interesting.

For future reference, don't forget IEEE Explore has a heap of conference proceedings online. Found the proceedings for InfoVis 2004 there. This conference sounds quite good. I've missed the deadline for 2005, but it might be worth keeping an eye on.

Had a coffee with Christopher Lueg and he pointed out that the approach I'm taking in my PhD is really participatory design. Even though I've actually used this term before in papers, etc I'd kind of forgotten that this is an entire area of research which I can use to justify my approach. Should: a) read about this; b) talk to Toni.

friday april 29, 2005 (part II) - trip reflection

Decided to split today's entry into two sections, because I wanted to write something about the PEP trip. I think today I'll jot down some quick notes without worrying too much about what happened when, etc. just so I capture as much as possible before the memories start to fade.

First up, even though it occured towards the end of my trip, the discussion with Geraint Wiggins was a rapid-fire and wide-ranging talk. The main points that stuch with me were that it is important for my research work to look at the work done on perception of musical elements. That is, when people discern a phrase as being 'legato', what does that mean in terms of the accoustic properties of the sound? While I can say that my spheres visualisation makes it obvious when you are not playing smoothly, it would help my case if I could say, "X has shown that consecutive notes played with a gap of less than 20ms between them are generally perceived as being smooth, therefore I have incorporated this information into my visualisation", or some such.

He also had an off-the-cuff suggestion that perhaps I might use some kind of 'genetic' algorithm to generate the mappings between sound and visuals. The idea would be that the algorithm generates two mappings, we allow a musician to use the resulting visualisations and then tell us which was more effective. The less effective one is 'killed off' and another generated based on the more successful mapping. Obvious downside is that there is an absolutely huge number of mappings to try and it would probably take too much time to explore all these possibilities. The idea is worth keeping in mind though, as it's a way of making the process of generating visualisations more scientific and less artistic - which might be a good or bad thing perhaps...

Geraint also suggested some good sources of material. ESCOM- can't find what this stands for, but it's a psychology/music conference which runs every 3 years. Proceedings for the 2003 conference are here. He especially mentioned Patrik Juslin, and there are a number of (short) papers by Patrik in the proceedings. The Feel-Me project, which is looking at defining the nature of musical expressivity and finding strategies for teaching expressive playing. The second part sounds more fun than the first. Anyway, this is obviously an important project I need to look at.

friday april 29, 2005 - interesting paper

After receiving the table of contents alert for the Journal of Research in Music Education I found a nice paper on the use of 'figurative' language in music instruction. Figurative language means metaphors, similes, analogies and visual images. The paper itself was interesting in that it examined the effectiveness of 'informal' figurative language ('like popping popcorn...' for example) versus the use of more the formalised figurative language used in standard music notation, such as 'giocoso'. Perhaps more useful for me though is the literature review which includes what sounds like a really nice justication for my work by Barten (1992 & 1997). The paragraph reads:

Figurative language in private instruction, chamber, and large groups was reviewed. Barten (1992) contended that in music instruction, standard descriptive musical terminology inadequately captures true musical essence. She focused on idiomatic metaphors ("closest to literal expressions," p. 3), nascent metaphors ("they only implicitly refer to a second domain; they still appear to be rather direct descriptions," p. 4), and explicit metaphors ("involve a more explicit indication or evocation of a nonmusical domain. Sometimes these metaphors are quite original and even poetic," p. 4). After she observed and interviewed college music students and faculty in various rehearsal settings, Barten reported on the classification and frequency of metaphors that emerged from each setting, finding a prevalent use of explicit and nascent metaphors. She speculated that music instructors might choose to use metaphors because of a perception that it is a more effective technique in teaching expression compared to other methods. Barten (1997) extended and specified her argument, maintaining that the most effective and frequently used figurative language revolves around motor-affective properties of music. Instructors tend to focus on aspects of the music that have to do with momentum and the description of certain movements. She suggested that the style embodied in creative figurative language may both assist instruction and engage the student. It may serve as a source of intrigue, be poignant and memorable, "hit a nerve or strike a chord." It was proposed that music teachers rely on figurative language for lack of a more direct approach.

The two references are:

• Barten, S. S. (1992, February). "Like a single crocus in Holland": The power of metaphor in music instruction. Paper presented at the Second International Conference on Music Perception and Cognition. UCLA, Los Angeles.
• Barten, S. S. (1997). Speaking of music: The use of motor-affective metaphors in music instruction. Journal of Aesthetic Education, 32 (2), 89-97.

Anyway, the research reported in the paper was an investigation of whether the informal figurative language was more or less effective the the formal figurative language when used to give instructions to musicians on how they should play a piece of music. So, some musicians were asked to play a short phrase "like popping pocorn" for example and others were asked to play "staccato". The effectiveness of the performances were then evaluated in terms of how well listeners matched the performers to the formal and informal figurative descriptions. The methodology was more sophisticated than this, but this is what it boiled down to. I'm not sure that the aim of playing music is necessarily to transmit an emotion from composer to listener, nor am I sure that there is a "correct" or "successful" interpretation of a performance, but this may be being unfair. Regardless, for the purposes of this study this was the assumption.

thursday april 28, 2005 - gestural interaction workshop

Still to write reflection on trip. Anyway, yesterday attended the gestural interaction workshop at NICTA in the Australian Technology Park (where the Con. used to live). Well worth attending to see the research issues and state of the art in this area. It's interesting that almost no matter what research area you look at, the fundamental issue of context keeps cropping up. To what extent can you ignore context (in an area such as gestural interpretation for example) and still produce research of value? The keynote at this conference acknowledged this issue, while suggesting that there was still plenty of 'low hanging fruit' for the picking without necessarily confronting the big questions. That is, it is possible to produce computer systems using gesture tracking (for example) which people find useful without necessarily solving the philosophical issues at the core of the 'meaning' of gestures. (And the meaning of meaning...)

tuesday april 26, 2005 - live algorithms for music network

Well I'm mack from my trip and will add comments about how it all went shortly. In the meantime, Tim Blackwell, who I met at the CC2005 conference signed me up to his mailing list/network, live algorithms for music (LAM). Tim has examples of his work on his website and he's keen to work with me on a project which uses the "spheres" visualisations and adds some kind of musical intelligence to several chains of spheres.

sunday april 3, 2005 - sporadic update from the trip

I have some time to update this blog now as I'm in the Ars Electronica center cafe which has internet. Ars Electronica has been pretty cool and has made me realise that there really is a lot of potential in this kind of technology and that it can only get more amazing. Nothing here is really like what I think I want to do with music/pd, etc although there was one similar exhibit which reacted to your voice and put "bubbles" on the screen based on how you sang. On another setting you could 'draw' with your voice and a third setting showed a kind of big bubble around a silhouette of your head which expanded and changed shape based on the quality/loudness of your voice. I took a photo, so hopefully should be able to check out who designed this, etc.

While I'm talking about things to do, I need to remember to check out the group in Switzerland that Werner from Simon Dixon's group mentioned. Apparently they had a project with the same concept as mine, or at least similar.

I should also come up with some ideas for experimenting with Simon's toys.

I'm thinking my PhD title might perhaps be, "Visualisations of music for use by instrumental musicians". This is because I'm not intending to do low-level work developing new algorithms for beat tracking or sound analysis but rather am investigating how to support real musicians using technologies that exist. It's how the technologies are applied and presented that is interesting I think.

hmm...think maybe I'll see if I can hire a bike to ride along the Danube. On the other hand, it's bloody cold :)

tuesday march 22, 2005 - reflective practitioner

I need to check out The Reflective Practitioner - How Professionals Think in Action by Donald Schoen before I write the creativity and cognition presentation. There is at least one copy in the library. Also, remember to return Alastair's grid book..

monday march 21, 2005 - international conference on auditory display

Rodney pointed out the Interntaional Community for Auditary Display is an interesting group. Proceedings for their conferences are online. Some interesting looking papers in the 'audio-visual worlds' area.

Had meeting with Dianna Kenny this afternoon. She seemed quite interested in what I'm up to, especially in the use of repertory grid to explore musicians' attitudes to music. She was keen for me to incorporate some form of more 'traditional' research and drew up a rough plan of an evaluation of the proposed tools. The plan would be to start with a pool of musicians meeting particular criteria (eg. AMusA or above?). These people would be randomly assigned to 2 groups, one of which is a control group. The non-control group would be given an into to the tools and be given an opportunity to play with them. Then they would be given a piece to learn which incorporated aspects that were designed to be addressed by the tools. The control group would be given a lecture or brochure or some such which took the same amount of time as the intro to the tools and would then learn and perform the same piece as the other group. Recordings of both sets of musicians would be randomised and evaluated by a panel of experts.

Booked Hotel Plessis, Address : 25 Rue du Grand Prieur� 75011 - Paris, Email: plessis@venere.com, Fax n.: +33-143579787, Phone n. : +33-147001338, 63EUR per night, booked through venere.com. Reservation request code: 1446161.

friday march 18, 2005 - more trip...

The Asakusa View Hotel is an expensive but low-stress option for Tokyo I found in the lonely planet. Can book online.

But, after talking with Jim decided on Hotel Green Capital. Booking details here.

Sent request for cost to Hotel-Pension Golz in Frankfurt. Costs are from 45-65EUR for single.

thursday march 17, 2005 - more trip...

Sent email requesting booking for Frhstckspension Wilder Mann in Linz which looks ok and is 32.00EUR per night including breakfast.

Good website for Karlsruhe accommodation.

Booked Residenz Ringhotel Karlsruhe, Bahnhofsplatz 14-16 76137 Karlsruhe; Tel: +49 (0) 721-3715-0, Fax: 0049 721 3715 113, EMail: info@hotel-residenz-ka.de for ONE night only April 4, 41.00EUR (figure I'll travel to Frankfurt late on the 5th).

Booked train London-Plymouth return via Virgin Trains: Fast Ticket collection from LONDON PADDINGTON station your FastTicket Reference is C88F8R5F.

wednesday march 16, 2005 - repertory grid stuff

The presentation on rep grid for pub club yesterday went ok. Also, Alastair interviewed me using rep grid as part of his research into what effects peoples perceptions of elements of peoples environments (ie. desks, offices) have on their perceptions of that person.

As far as reg grid software goes, so far I like RepGrid IV, but even though the (free) personal edition will probably suffice for my purposes, if the full version is required it's very expensive. As Ernest was interested in the link between rep grids and concept maps, it was interesting to see that the manual for RepGrid IV discusses this and mentions that they are "complementary" approaches.

Element elicitation brainstorming: What things have you done in your practice that have made you notice new things about your playing? Tell me things you have worked on that have helped / not helped. What are things that have motivated you in your practice?

For the record- booked Victoria Court, 62/64 North Rd East, Plymouth, Devon PL$ 6AL, tel: 01752668133; check in 6/4/05 for 3 nights; 42GBP per night.

For cheap hotels in paris: eurocheapo.com! Hotel de Nesle looks promising...

monday march 14, 2005 - repertory grid: principle component analysis

Found this tutorial on principle component analysis, which is referred to quite a lot in rep grid literature.

friday march 11, 2005 - more trip planning

Noticed that IRCAM has a cool-sounding conference, the voice, on Monday April 11- my last night in Paris :)

thursday march 10, 2005 - more trip planning

British National Rail website allows you to book/check fares online.

wednesday march 9, 2005 - physics processing unit / trip planning

Noticed on slashdot that a physics processing unit has been released. Interesting that games, etc are moving in this direction. Makes my work with pmpd seem quite on the cutting edge..

Did a trial rep. grid interview with Rob Kay on approaches to music practice. I think the best way to elicit constructs will be with a question such as, "Tell me about a way in which two of these elements are similar in terms of the outcome you're aiming for while playing them".

Doing some trip organisation. For reference, amount of PEP money received: $4,710.00; Amount of seed funding remaining: $2,904.90; Airfares: $2,478.89; CC2005 Registration: $423.00 (approx). Total amount available = $7614.90. Amount remaining = 7614.90 - 2478.89 - 423.00 = $4,713.01.

So, for 25 days that leaves 4713.01 / 25 = $188.52 per day which has to include all the ground transport!

Booked Vienna hostel: Hotel Geblergasse (via hostels.com; Address: Geblergasse 21, Austria; Phone Number: +43 1 4063366; Fax Number: +43 1 2836462; Email Address: hotel@geblergasse.com; booked for 5 nights, arriving Mon. March 28, departing Sat. April 2. Total cost: EUR214.50, paid deposit EUR21.45, remaining EUR193.05.

For booking in Linz, found the Linz Tourism site, which has some nice-sounding hotels at reasonable prices. Seems better than the dodgy booking agency websites...

For Karlsruhe, the old Let's-Go Germany suggested Pension am Zoo, which looks reasonable.

Also, might be possible/better to go from Karlsruhe to Plymouth by train, as it's not that big a trip to Calais... See Eurail for details.

tuesday march 8, 2005 - ethics application approved

My ethics application for the repertory grid interviews with musicians was approved at the CCS meeting this morning :)

Email from Springer advising that proceedings of Computer Music Modeling and Retrieval symposium in Denmark are available. Don't know if it's worth 54 euros, but noticed that last year's proceedings are in the library. Another thing to add to the list..

monday march 7, 2005 - back after mini-break

Mum and Dad came up to visit Thurs-Sat last week. Was great for Zoe to become reacquainted with her Nana and Grandpa and she seemed to really have a ball.

Back to work today: reading Susan Hallam's chapter in Does practice make perfect. Useful note for future reference: Paper by Cantwell and Millard in British Journal of Educational Psychology sounds like it could be useful in tying Jacob's approach to Biggs ideas of deep and surface learning. (See p. 90 in Hallam's chapter for summary.)

After reading Hallam chapter I'm thinking that elements for grid could perhaps become something like: "Strategies you have used in practice". Hmm..or maybe something along the lines of "tools in the toolbox"?

The British Journal of Music Education is probably worth keeping an eye on...

wednesday march 2, 2005 - in touch with Dianna Kenny

Dianna Kenny responded to my email very positively. She has just submitted an ARC Discovery grant application on the topic of practice, which is pretty cool so hopefully we will be able to work together. We set up a meeting for March 21 when she returns from an overseas trip.

Otherwise, spent today working on RMIT application and a todo list:

• ethics app.
• contact Kate Stevens
• work on CC2005 presentation
• repertory grid presentation
• advise Jim of PEP changes
• collect ticket
• join interaction design class?
• rmit app

tuesday march 1, 2005 - ccs doctoral assessments

Went to Lizzie and Yun's doctoral assessments today which brought some interesting stuff to my attention. In particular, Lizzie spoke of David Rokeby's idea of interactive artworks as "transforming mirrors". I think this is a nice metaphor for what my exploratory music pieces are about. Need to look into this..

Also, for the record a reference for the 'think-aloud'/retrospective reporting type research technique is Ericsson and Simon.

sunday february 27, 2005 - IRCAM

Kirsty Beilharz suggested that forumnet is a good place to look for info on IRCAM software/research groups. Once I have a clearer idea of who might be best to contact she will help if possible..

Ok, and while browsing forumnet, I found OpenMusic which looks interesting. Found a summary here.

I wonder how they go about deciding what their software should do?

saturday february 26, 2005 - people to meet...

Gary McPherson responded to my emails and suggested that a "prominent music psychologist" is likely to be appointed to the Sydney Con Applied Research department and that it would be worth making contact. Also, he suggested that Kate Stevens at UWS would be worth contacting.

friday february 25, 2005 - history of interactive media

Greg found an interesting and humorous history of interactive media which I read while cooling down after the ride in.

While looking at Geraint Wiggins' home page I came across the Music Mind Machine group at both NICI, University of Nijmegen and ILLC, University of Amsterdam. It's a little unclear what the status of this group is, but could be worth visiting?

Looking into concept maps to figure out their relationship with repertory grid. Found an overview of the underlying theories of concept maps.

thursday february 24, 2005 - lehmann chapter on mental representations of music

Finishing reviewing the chapter in Does practice make perfect Jorgensen and Lehmann (eds.) by Lehmann on "acquired mental representations" of music. As part of the lit review he outlines three pedagogical approaches to conceptualising music: visualising (Leimer-Gieseking method), Audiation (Edwin Gordon's method) and the "photographic ear" (described by Macek (1987)- similar to photographic memory. Somewhat pooh-poohed by Lehmann). Should follow up on these leads :)

Also, met with Ernest today to give progress report and make some plans. His suggestions:

• Contact Kirsty Beilharz to ask about visiting IRCAM. She has been on a scholarship there and should be able to help with contact details.
• Contact Geraint Wiggins who is at Goldsmith's College London. He would be good to visit while there.
• Checkout Kumiyo Nakakoji's home page as a starting point to look into Concept Mapping and how it relates to repertory grid.
• Prepare full ethics application for next ccs meeting.
• Write reflective account of Ben project. (How has it influenced my thinking?)

In terms of grants, he mentioned that in addition to ACID and the Australia Council it could be worth looking at pursuing an ARC International Linkage Grant and/or an International Science Linkage (ISL) grant. The idea is that you work on a substantial project with someone from an overseas uni. They might say that they will pay your hotel bills while you are visiting them and, for example, Jim agrees to let me work on the project for 1 day per week. This means that the international uni is contributing accomodation costs (say a few thousand dollars worth), UTS contributes my salary for one day per week (few thousand dollars) and if you get the grant the government kicks in an equal amount in cash. His suggestion was to raise the possibility of doing something like this with people I meet on my trip.

wednesday february 23, 2005 - pmpd

Did some more reading on physical modelling for pure data (pmpd) which I like as a way to incorporate "natural" effects into computer music/tools. Found this paper by Chris Chafe which gives an into to some compositions using this approach. Also his home page has some interesting stuff and could be worth checking on every so often..

Been annoying me for ages that I can't remember the ccs wiki link.

tuesday february 22, 2005 - gary mcpherson, etc

Sent an email to Gary McPherson to ask if we can meet up an discuss my research. According to the USyd website he is still on leave, but hoping he might be back this semester.

Alastair mentioned he had borrowed A manual for repertory grid technique by Fay Fransella, Richard Bell, and Don Bannister from the library. I should check this out when possible.. Also, Robert Kay suggested a book with a title something like Transformative Education by Pope and Denicolo. Haven't been able to find it yet.

Another thing: found GEM developers mailing list.

monday february 21, 2005 - back from holidays : grants

Well, back from a week on the beach at Huskisson :) and catching up on emails, etc. Ian posted an interesting call for works to the ccs-list for THE RESONANT IMAGE : Contemporary Graphic Scores and Performances. Might be able to submit some kind of virtual environment? Also, the stasisfield site has some interesting stuff available for download which I should check out..

Checked out the Australia Council web site this morning and noticed that details of grants for 2005 are now up. In particular I think that perhaps the Ben project could be elligible for a Music New Work grant, a New Media New Work or maybe a R&D grant in the new media category (although given recent restructures of the council this may be problematic). Closing date is the 1st of May for most of these.

A note for future reference: Imogen says seed fund balance remaining is $2904.90.

thursday february 10, 2005 - IRCAM

Checked out IRCAM's website this morning to see if I can organise a visit during my PEP trip. In particular, the Music Perception and Cognition, the Real-Time Applications research group and the Performing Arts Technology Research Team sound like they could be worth visiting. Finding email addresses is almost impossible but I managed to find Norbert Schnell's email via google.

I think that score following is an area that I should look at.

Roman recommends visiting ZKM, the Center for Art and Media in Karlsruhe.

Some cool electro-acoustic music I found when looking for info on granular synthesis is here. I especially like the one with double bass..

wednesday february 9, 2005 - electronic music page...

Reading the pd mailing list and came across disquiet, an interesting site devoted to electronic music which could be a good departure point getting into this stuff.

tuesday february 8, 2005 - microsoft and more reading

Went to MS 'Security Summit' this morning at Darling Harbour. Made contact with John Warren and Pradeepa who are the academic contacts these days. There is a potential to work with MS Beijing on research, although I'm not sure that they would be interested in CCS-style stuff. The most likely sounding research group there (Beijing) was the Interaction Design group who seem to be interested in prototyping and the link between design and computing.

On the lit. review front. I found the proceedings for the International Conference on Music Perception and Cognition 2004. The UI is super-painful (hint: make sure pdf view is set to 'continuous' view when looking at the contents and zoom right out).

monday february 7, 2005 - for the trip...

Should visit Ars Electronica when in Austria.

Also, Miller Puckette's draft book looks like it could be pretty handy.

Just a reminder: today I placed an inter-library loan request for "Does practice make perfect" by Jorgensen and Lehmann. It's in the Sydney Conservatorium library. The request number is 42043.

thursday february 3, 2005 - some more reading on approaches to practice

Library has access to the Psychology of Music journal (recent issues only). I'm starting to look more for articles which look at the strategies musicians use when practicing. In particular I'm obviously interested in any mental visualisations which they might use.

In looking throgh Parncutt and McPherson (Science and Psychology of Music Performance) I found the chapter on Practice by Barry and Hallam. There are several interesting sounding papers by Hallam that I should read.

wednesday february 2, 2005 - started drafting investigation plan

Feel like today was reasonably productive. This morning worked out a structure for a paper for Julie to submit to the Agile 2005 conference and this afternoon started drafting a plan for my PhD investigation. Basic idea now boils down to interviewing musicians/teachers to find out what (if anything) they think about (or visualise) when playing. The idea I have at the moment is to interview them but also get them to record their thoughts as they practice over a certain time period (a week say). A nice way to do it might be to give them mini-recorders for that time and then when something occurs to them they can say it into the recorder and perhaps also record a snippet of audio to illustrate if necessary. I don't think many muso's would like to be 'observed' during practice and I don't know what this would achieve in trying to get a picure of their mental processes, so perhaps this is the best option. I need to talk with Ernest and maybe Shigeki to see if they have any ideas about this.

tuesday february 1, 2005 - pd gui stuff

These pd gui objects look like they might be useful.

Don't forget- tomorrow: checkout the Australian Music Centre and try to find the "newmusicnetwork" website...

monday january 31, 2005 - gem tutorial

Checking out the pd site and found a GEM tutorial that I don't recall seeing before. Pretty basic, but could come in handy.

sunday january 30, 2005 - ben project brainstorming

Some more ideas...

First misterioso section ('The Juggler')

• Background audio snippets- patch records many (30?) very short snippets of audio (triggered by volume? or maybe random?). These are then played back to create an 'audio cloud' effect.
• Balls in the breeze are now black (ie. indistinct/invisible). Movies or images textured on the balls flicker on/off making the balls more visible.
• Ball which is being moved by currently playing note/noise has most intense flickering. ie. If a D is played the second ball is most visible as well as having force applied.

Second intenso section (Circus movie?)

• Long notes cause movie playback to slow down. ie. Movie is playing back at normal rate. Volume controls 'flicker' or transparency. If volume remains high (ie. long loud note) then movie playback rate gradually slows right down.
• Eventually movie either freezes or loops on short section of movie. Then, title for next section is shown and finally, the 'rain' starts (see below).

Third 'rain' section ('The Magician')

• Flickering movie from intenso section gradually becomes more stable (requires louder and louder notes to cause flicker).
• 'Raindrops' of image start to pepper movie.
• Trombone pitches control where rain is most intense on the movie.
• Eventually, raindrops reveal complete image, obscuring movie completely.

Final, 'pink' section ('The Strong Man'?)

• 'Brutal' notes played by trombone cause big, solid blocks of pink to obliterate image revealed in third section. Blocks of pink may be either solid colour or 'speckled' white-noisy type blocks.
• Only quite loud notes trigger pink blocks
• Loudness over threshold sets transparency of each block. ie. A loud C creates a block in certain position which is semi-transparent. A very loud D creates a block in different position which is less transparent than the C.
• Grid of blocks is (probably) 6 x 4. ie. Need 24 loud notes to block out image completely.
• As each pitch is played, the computer recalculates which pitches relate to which remaining squares in the grid. ie. A note above the threshold will always result in a block appearing. This should hopefully prevent the situation occurring where the musician has one block remaining to fill in, but he can't get the computer to recognise the pitch to fill it!

Epilogue to Strong Man...

• Now that pink screen is showing, with some blocks semi-transparent. Ben starts a drone (on A). Computer records drone and starts to play back.
• As drone progresses, all blocks become completely opaque.
• After some time, very small variation in pitch (and volume) from A (microtonal variation) by performer triggers introduction of lime green (?) to middle four blocks.
• After pause, pink screen fades and balls in breeze reappear, but with one ball coloured lime green (or pink?). That's it :)

saturday january 29, 2005 - ben project to do's

List of things to do for the Ben project...

• Be able to have multiple 'bracelets' of spheres for the 'balls in the breeze' section
• Once audio playback has been triggered, it will always finish playing before allowing playback to start again. Same for record.
• Videos on the spheres - should be flickering and similar hue (probably b & w / sepia)

friday january 28, 2005 - more pd progress

Don't know why I haven't done this before, but found and installed the Pure Data Documentation Project files. Basically seems to replace many of the standard help files for basic pd objects.

Getting frustrated trying to get pd/gem working properly in the lab. It runs and stuff does work, but really it's quite crashy. Perhaps this is a g5 thing?

Anyway, should start to make a list of things to do with Ben...

• calibrate fiddle~ to figure out which settings give the most accurate results for trom.
• get some video footage to use
• run through spikes
• show pmpd stuff
• remember fonts are possible

wednesday january 26, 2005 - more pd progress

Starting to make faster progress with pd. Finding useful libraries such as maxlib which include beat trackers, etc as well as many more mundane but useful objects such as averagers, scalers, and so on.

Also, the fftease library looks useful. Apparently includes a phase-vocoder for speeding up/slowing down audio without changing pitch and other fft objects.

monday january 24, 2005 - java music again?

Still avoiding writing this paper. Found interesting page: jMusic - a set of java class libraries for doing music stuff co-authored by Andrew Brown who's been to visit CCS as an ACID rep. Not sure if this will be useful or not yet...

On another matter, the dynamic creation of objects in pd. Rather than clone, perhaps dyn~ could be better.

friday january 21, 2005 - color organ

While supposedly working on a paper co-authored with Alastair, I started reading Linda and Ernest's book- specifically the chapter by Jack Ox on the color organ, which appears to be a substantial work in the area of musical visualisation. Also it would seem to be an enormous programming challenge/achievement! A relevant point is probably the mapping described in that page between pitch (y-axis), attack-volume (z-axis), time (x-axis) and change in volume (color). Another interesting point (for Ben's project?) is that the viewer can move through the 3d world created and will hear the music at the point in space-time that they are currently at. Perhaps for our project certain musical elements could trigger movement backwards in time, effectively 'rewinding' the audio/visual component and giving the performer the opportunity to 'change the future!'

Ok- grant/industry link idea: Get some 3D animation company interested in creating 3D musical worlds for musicians. My PhD provides data on what how musicians' think about while playing and how they conceive sound mentally and they provide expertise in modelling/3D programming, etc. Could be a 3-way link between artists, industry and academia. I like the sound of this..

thursday january 20, 2005 - useful pd thing

Found the need to create multiple instances of GEM objects for 3d visualisations. Clone looks like it may do the trick.

wednesday january 19, 2005 - googlings of 3d visualisation of music

Googling around for 3d visualisations of music and found Interactive Visualisation of Musical Form using VRML by Alan Graves, Chris Hand and Andrew Hugill at De Montfort University, Leicester. A really nice description of the basic principles of attempting to visualise music in meaningful ways. (Apparently this is from Organised Sound Vol. 4i pp. 15-23 -Cambridge University Press)

Also, must remember hyperinstruments for the lit. review...

Journal to check out: Organised Sound. This has the Graves, Hand and Hugill paper mentioned above.

Stumbled on interesting blog-style report on live club visuals which talks about many software packages and tools for VJs to use to produce live video for clubs. Also has a lot of potentially useful links in the appendix.

Really nice site: soundtoys. Kind of like rhizome.org, but for interactive sound work; it showcases the work of artists working in this area.

Possibly interesting paper: Music Learning through Visualization by Hiraga, Watanabe and Fujishiro. May touch on an area I haven't really thought much about- computer support for ensembles. Will read sometime..

monday january 17, 2005 - sick of pd

Well, I suppose I've made some progress over the last couple of days. The Physical Modelling for Pure Data library is very cool. It allows you to draw objects with gem and use pmpd to model their interactions based on Newtonian physics. This makes for some very 'natural' looking movements. I hooked them up in a simple way to pitch analysis using fiddle~ and the results do look pretty cool.

Also, started to look at pitch shifting as this is something that I think Ben was interested in. Unfortunately (so far) even though the pd fft examples do include a pitch-shiter, it's nowhere near as nice to use as the Jehan external I found for Max/MSP. Starting to feel that this whole episode is taking rather too much time!

friday january 14, 2005 - trip, and more stuff to read

Was getting organised and teeing up dates to visit Eduardo Reck Miranda and started surfing around his website. First, found a really nice chapter in an upcoming book, Musical creativity: Current research in theory and practice by Irene Deliege and Geraint Wiggins which I need to keep an eye out for. Somehow I then ended up finding Jeanne Bamberger who has written some very interesting sounding books, the latest of which is Developing Musical Intuitions: A Project-based Introduction to Making and Understanding Music- yet another book I really need to track down. The more I look the more I find...

Also, stumbled onto the PsyMus mailing list. Looks a bit quiet, but may be useful.

hmm...another interesting project- ChucK- an audio programming language using C-type syntax.

tuesday january 11, 2005 - later

Well got pidip to work after much stuffing around with fink and X11, etc. I can't see that this is going to be a workable system. All the X11 windows all over the place running within OS X just looks dodgy. In addition, so many of the objects seemed flakey and crashed often. Maybe I will feel better tomorrow, but so far today feels like a big waste of time.

tuesday january 11, 2005 - catch up

I think it's time to catch up on news, seeing as it's the new year. Holiday in Melbourne for 10 days or so staying at mum and dad's which was great- especially for Zoe who had a ball getting to know everyone a bit better. Still trying to think of a way to get back down there, but with the way IT faculties seem to be going that's not going to be easy.

As far as PhD stuff goes, I've been struggling with pd and making some progress but it is a very slow process. The problem I was working on was Ben's idea for a first 'variation' for the project we've got in mind. The idea is that there is a movie playing in the foreground hiding a picture behind it, and as the musician plays each note "pokes a hole" in the movie so you gradually get to see the background picture. Like a good IT person I thought this would not be too tricky and, as it usually does, it turned out to be far more complex.

The library used for graphics programming in pd is GEM (Graphical Environment for Multimedia) which is basically a front end for OpenGL the 3D graphics programming language. GEM provides a nice layer on top of OpenGL to save you having to get too low level setting vertices of triangles, etc. I can see there are a lot of possibilities for using 3D space for musical environments.

So I spent a lot of time looking at the GEM examples and so on and generally getting to know it. However, plotting points from a picture on top of a movie turned out to be far more complicated that you might think. Showing a movie is trivial and showing a picture is trivial. Plotting points from a picture one at a time on screen was not too difficult and two of the examples (04.pix/10.PixDataSimple.pd and 04.pix/11.PixDataComplex.pd) pretty much do this (although for some inexplicable reason on the powerbook Mac the pictures are upside down) but because you need to use single buffering when plotting individual pixels (well acutally they are very small shapes rather than pixels) combining this with video playback is complex. Video playback uses double buffering, where the frame currently being shown is in one buffer while the next frame in the sequence is being prepared in the other buffer. This is to remove flickering. The problem seems to be that the pixels drawn in one buffer are wiped out each time the buffers switch over. I assume that pix_film, the video playback object then clears the background buffer before uploading the next video frame. Thus the dots appear on the screen very briefly but do not persist as we want them to.

So, that's where I'm at today. I think I'll look into another possible solution using pix_rectangle for drawing, but I suspect I will have the same problem with buffering. Perhaps I can redraw all the 'activated' dots every frame??

Another possibility is to write an external (or modify one of the Gem externals). I looked at pix_set which is promising because you can send it a list of RGBA values to display, but what I need is to be able to modify those pixels I want to be transparent. ie. It would be good if I could get access to the video frame data and change certain pixels alpha values to allow them to become transparent. The thought of trying to compile Gem is frightening though.

...oh another track: try Pure Data Packets (PDP) and/or Pure data is Definitely in Pieces (PiDiP??). See these install notes for PiDiP.

tuesday december 14, 2004 - more pd

Making (slow) progress. Have now found a new 'experimental' version of pd on Miller Puckette's site which I've installed on the work PC. Looks like this version is also more OS X friendly as it has a .app installation which doesn't need to put files in weird places in usr/local/... Installed the latest version of GEM at work too, and set the 'load at startup' option for PD to load Gem.dll. Starting to get the hang of the environment anyway. Also, got an avi movie to play using Gem :)

sunday december 12, 2004 - more pd stuff

More work getting pd working with video stuff. Apparently the packages in this area are GEM for 3d graphics and PDP for video. Gem is working now (with the -lib command mentioned below) but PDP needs some extra libraries installed for it to work (I think). A recommended way to get these packages is using fink, a tool for OS X which makes installing various unix-based packages simpler. Working on this now...

Another useful pd link: the pd list archives.

saturday december 11, 2004 - more pd stuff

Found some more useful sites for pd. The pd webring has a lot of good links to start with.

Also downloaded GEM from here. Thought this was included in the main download of pd?? (Answer: it was, but this is a newer version which is good anyway.) Also, to make sure GEM loads you need to add "-lib /usr/local/lib/pd/extra/Gem" to the pd command when starting pd. On the powerbook I've put this line in a .pdrc file in my home directory which means it is always loaded when pd is run...

thursday december 9, 2004 - more books...

Just an entry to save the details of the local distributer of MIT Press and other publishers, Astam Books. Name of academic rep is apparently Bob Wilson who can be contacted at info at astambooks dot com dot au.

thursday november 25, 2004 - more audio programming stuff

Found a nice-looking intro to digital signal processing which I hope to get time to check out today.

wednesday november 24, 2004 - jMax more

Actually the tutorial is pretty useless as it's incomplete. A better reference site could be this quick reference. Still looking into this..

Ok, jMax could be good. Also looking into just using Java to write sound stuff. This site looks like a good starting point...

...and on reading that, perhaps the Java Media Framework could be a better way to go. It's more high-level than the Java Sound APIs apparently.

(BTW- good funk compilation found in Greg's mp3 collection: Pulp Fusion. Think this is a series of CDs.)

tuesday november 23, 2004 - jMax

Been doing some reading on jMax today, which is basically another branch of the Max/MSP tree similar to pure data. It's open source which is good and the UI stuff is written in Java. I'm not sure though about the availability of externals, etc, although I like the idea of being able to write externals in Java (if this is actually possible)...

Anyway, reading the tutorial this afternoon.

tuesday november 16, 2004 - grants

Pauline has done a quick search for possible grants and come up with the following starting points. Need to review these, check with Ben about other possibilitites and do a search of my own.

monday november 15, 2004 - meeting with Sid Fels and Sachiyo Takahashi

Had a really interesting meeting with Sid and Sachiyo this morning and discussed their work and my PhD. He mentioned a project he had been involved with - the MusiKalscope, which combined the Iamascope and a system for supporting jazz improvisation.

sunday november 14, 2004 - repertory grid theory

Robert Kay mentioned Repertory Grid Theory, which sounds like a nice way of structuring interviews in order to avoid the interviewer introducing bias. Basically the interviewer shows the interviewee three cards showing three aspects of the area being investigated. For example, it could be the name of three people. The interviewee is then asked to think of some way that two of the elements are different to the other element. ie. In what way are two of these people the same in contrast to the third person. This technique can be used to elucidate how the interviewee constructs their view of the world. The data is put into a grid for analysis I think which is how the name came about. Anyway, the link above seems to give a good overview, so I'll work through it...

friday november 12, 2004 - NIME proceedings, etc

Found the proceedings for New Interfaces for Musical Expression (NIME 2004). Particularly like the paper about heating up piano keys to stop people playing wrong notes!

wednesday november 10, 2004 - journal to target?

The Journal of Technology in Music Learning could be worth submitting to. Should also organise for the library to subscribe.

wednesday october 20, 2004 - some stuff from Peter Gale

Peter G mentioned a linux distribution or set of packages - Planet CCRMA ("karma"). Also seem to be a lot of people at stanford doing interesting music related research. Could be worth a visit?

Also, the LABS, Leonardo ABstracts Service, containing PhD/Masters abstracts in the artsy field could be useful.

thursday october 7, 2004 (later) - other news

Lots of other stuff happening lately which I have been slack about reporting on.

• Zoe is growing fast- big baby: 8Kg and 67cm which puts her at the very top of the charts for height and weight. She had a fun time in Melb. when we visited last week and got to catch up with everyone. I think she really got a lot out of meeting so many people and being out and about. Flying was ok, although she got slightly shirty at the ends of both flights.
• ACIS paper accepted :)
• Wrote a new paper with Shigeki. Similar to OZCHI but with discussion of music as a 'design problem' and report on Shigeki's drum project.

thursday october 7, 2004 - finally figured out 'resource-forks', max msp and CVS...

Finally figured out how to get around the issues that CVS has with resource forks and OS-X. The only work around seems to be to use MacCVSClient which seems a reasonable cvs front end for OS-X which has two additional ways of storing files: the data fork and resource fork in AppleSingle format and the "mergable resource fork" in RBL format, which should be mergable, diffable, etc. I didn't bother trying the RBL format as the AppleSingle worked fine to store Max patches. After checking out all was well. To configure to handle max files correctly automatically, go to preferences-translators and add a filter: mask: *, creator: max2, type: *, storage format AppleSingle Data + Resource Fork (ASG), keyword sub: -kb. This should be enough, but may need to also add filter to Type: maxb if * causes problems or add in a *.pat filename filter if necessary...

friday september 24, 2004 PM - planning for pep

I think I should visit Simon Dixon at the Intelligent Music Processing Group at the Austrian Research Institute for Artificial Intelligence in Vienna. This is one of the guys who wrote the "worm" visualisation...

friday september 10, 2004 PM - reading

Shigeki found an interesting visualisation paper: Visualization of Music Performance as an Aid to Listeners Comprehension by Rumi Hiraga and Noriyuki Matsuda. Discusses a way of viewing temp and articulation differences in a simple graphical display. Similar to the "worm" by Dixon, Goebl and Widmar (I think).

thursday september 9, 2004 PM - singing tree

Found an interesting project: The Singing Trees which were a part of the brain opera project by Joe Paradiso. William Oliver is the main artist involved with the Singing Trees which involved a video display and a microphone which users sing into. A computer analyses the singing and as the note becomes more stable it produces "angelic" accompaniment. If the note is unstable the accompaniment and video is more disjointed and dissonant. Apart from the artistic angle, this type of thing sounds like it would be great fun for young students to play with...

A PhD thesis referred to by Oliver is MUSICAL SOUND INFORMATION: Musical Gesture and Embedding Synthesis by Eric Metois. This apparently contains a DSP tool kit which can be used to analyse vocal (and other?) input.

Another useful link- Home page of Tristan Jehan, a PhD candidate in MIT media lab. Lot of interesting sounding papers to follow up on. Today I played with the "shifter~" pitch shifting object. Pretty cool.

tuesday september 1, 2004 PM - reading

Read paper by Rob Saunders and John Gero on "artificial creativity" (AC). It had a very good lit. review of creativity work, but it had a similar problem to some other AI-type papers I've come across in that it seemed somehow 'recursive' or something. They set up an AC environment incorporating agents (..) which sought out novelty to an extent based on some factor. The problem I have is that they set up this environment and built into it certain values perceived to be human-like with regard to creativity and then observe the human-like behaviour that results in the agents. What has this shown? That human-like behaviour can be observed in systems that obey certain mathematical rules. But surely, if there is an infinite amount of rules to choose from and you deliberately choose ones which you hope will result in this behavoiur this is not particularly useful..

Will think more on this...

tuesday september 1, 2004 AM - another brainwave

PhD idea: Development of multimedia musical pieces that encourage "creative play". Use Ernest and Linda's model of artist-technologist-observer to discover implications for design of creativity support tools/environments. This sounds like it would be fun to do and quite easy to get composers, musicians, etc interested and involved. Contributions:

• Insight into design of creativity support tools
• Development of new techniques/technologies including programming techniques, network applications, AI...
• Development of a methodology which encourages a creative approach to application development
• Novel approaches to "requirements gathering"

Does this idea have legs?

monday august 31, 2004 - creativity and cognition reading

Very slack about updating this blog. Stuff that's been happening:

• Zoe getting cuter
• ozchi paper rejected :(
• Been reviewing interviews with Ben and Ros in order to write report on what has arisen so far and outlining likely structure of future, more comprehensive, targeted interviews
• Lots of .net badness
• Saw really cool CD-ROM of William Forsyth from Frankfurt Ballet. It showed his approach to choreography using quite simple, but nifty "augmented" reality. ie. He described drawing a line using his arm and 'playing' with the line and a line was superimposed on the screen as he discussed it and moved it about using various dance moves. Similar approaches for music?
• Been worrying about PhD. I guess I imagine an 'ideal' PhD envrionment would be similar to my VCA degree- surrounded by people interested in the same things. Each day at the coal-face. At the moment there are too many distractions...

friday august 6, 2004 - zen and the art of motorcycle maintenance

Where does all the time go? Some interesting reading lately. To start with, Zen and the Art of Motorcycle Maintenance by Robert Persig. I've been meaning to get around to reading this for some time as many Chicago-style people have mentioned it. Basically it's an extremely readable outline on the author's obsession with "quality" and what it means. He argues that the traditional "scientific" approach falls down when it comes to quality as it can't be precisely defined or measured, and ends up proposing that quality is somehow independent of subject and object. Thus quality ends up being a kind of "life force" out of which everything else arises- which apparently is similar to a Zen Buddhist approach.

tuesday july 20, 2004 - max/msp todos

Fiddled with max/msp and in particular the fiddle~ object. Wrote a small patch to identify note names in audio input. Now I need to read about jitter to figure out how to map the recognised pitches on to the screen in a nifty way a la the CCS launch piece. The settings I used for fiddle were amp-range 60 60 and all default initial paramters. It seems to work fairly well (on the G5 mac anyway) but does not do so well for notes below a bottom of the bass clef g on trom. Need to experiment in an intelligent way to figure out what the best settings are for various instruments.

monday july 19, 2004 - brainwave

A thought for useful tool for practice/teaching. Tool records performance and displays it - perhaps in piano-roll/overview view. Then portions of the performance can be replayed, slowed down, graphically visualised, etc.

friday july 9, 2004 - possibly interesting proceedings and other thoughts

Technologies for Interactive Digital Storytelling and Entertainment and Adaptive Multimedia Retrieval conference proceedings could be interesting.

Also, more ideas for papers, etc. Classify functions for interactive musical environments. ie. Recording, playback, video recording/playback, still pictures recording/playback, gestures? Types of audio recording: unaltered, tonal processing (distortion, etc), musical processing (analysis of phrases- eg. inversion, harmonic and melodic modification). Types of musical processing: melodic, harmonic, predetermined, emergent. The idea is to summarise the types of interaction and then to justify choosing particular types for particular purposes.

Had a look at the MIT Encyclopedia of the Cognitive Sciences - it's been sitting on the shelf for a while.. Some great entries on motor learning and the differnce between "implicit and explicit" memory, which could be useful. Also a handy little entry on creativity.

wednesday july 7, 2004 - protype ideas

Working on paper for ACIS and been thinking about possible tools. Thought automatic accompaniment for orchestral excerpts could be good. ie. player starts playing an excerpt and computer recognises it after few notes and plays orchestra parts. Really only a convenience feature I guess...

sunday june 13, 2004 - reading on grounded theory

Started to think about summarising my research approach for approaching pub club presentation run by Linda and Ernest and found this useful link on grounded theory.

Reading back on notes I made when talking with Margaret Boden earlier in the year I realise there are some books I need to check out. Mainly: Virtual Music (MIT Press), Computers and Analysis of Musical Style (Oxford University Press). She also mentioned I should check out Christopher Longuet-Higgens ("Mental Processes"), Paul Hodgeson and the Prof. of Music at UCLA (or similar? University of California?) Davis. Can't recall in what context this was mentioned now.

saturday june 12, 2004 - changes!

Well, once again it's been a while and a lot of changes. First and most important Zoe arrived on the first of June. She's gorgeous and we're have fun adjusting and fitting in with her way of doing things. She's probably making some compromises too :)

Anyway, in addition I've been trying to finish off a paper for OZCHI and my PEP application which are both due at the end of next week. In the process of organising PEP I've been checking out various web sites of people that Ernest suggested it would be good to visit. One he didn't mention but who looks very interesting is Eduardo Reck Miranda, who seems to be into stuff with interesting cross-overs with my ideas.

In addition, downloaded Beyond Productivity, Information Technology, Innovation and Creativity which was available as a pdf for US$24.00. Looks like it should be quite useful.

What else? Completed and submitted my PhD application. All in all pretty busy really.

april 20, 2004 - machine musicianship

Well, been away for a while. Been reading Machine Musicianship by Robert Rowe. Fantastic reference for the various techniques available for designing interactive music tools/environments. Especially surprised at how much work has been done in the area of recognising musical phrases, etc using various AI software techniques. I like the idea of neural networks and other approaches such as genetic algorithms that 'learn' musical styles rather than simply searching for pre-set patterns. Will write more about this soon...

april 9, 2004 - more night-thoughts on artist-technologist stuff

ok. I think I'm getting bogged down here. I think I just had a simplistic view of the creation process. A slightly more sophisticated model is needed.

• Model 1- One person takes role of both artist and technologist. Examples: Leonardo painting Mona Lisa, improvising trombonist, etc.
• Model 2- Collaborative artwork. People with different skills work on different aspects of an artwork. One person in charge of artistic vision.
• Model 3- Collaborative artwork. As above but artistic vision emerges from collaboration between two artists with different skill sets. ie. Visual artist contributes ideas for layout of graphic display and technologist implements ideas. The implementation is not a purely technical exercise and significant aesthetic decisions will be made by the technologist.

I know I read something like this in Linda and Ernest's book. Will have to refer back to this. At least I'm starting to sort out my prejudices/ill-informed opinions!

april 8, 2004 - music analysis modules for pd and other random thoughts

Found interesting library for pd which provides some kind of musical analysis features. Will look into this further...

Other developments: proposed 'pub club' for CCS at the meeting which I need to organise. To run at 3.00pm on last Thursday of the month (probably). Also, CCS symposium will run in November (16-19) and creativity and cognition conference will run in March in London. Think I need to crank out some papers...

Reflecting on yesterday's thoughts... Ernest mentioned in the Greg's presentation something about film crews and collaborative creativity work and, of course, a film is as much a work of art as the Mona Lisa. So...what is the parallel with computer artworks being collaboratively created by an artist and a technologist? Perhaps because there is a defined school of film-making or at least conventions that are followed this allows the director (or the person with creative control whoever that might be) to delegate the 'technical work' to others? Nah- creativity is expected from people at all levels here- actors, makeup, etc...

hmm.. what is the problem I have with the artist-technologist model? Well actually all I really want to say is that I think that if you are going to be creating art in a medium, you need to fully understand the medium. If you delegate work to others I think you should understand what you are delegating. Maybe I'm just a nazi. I guess in user-centered design the aim is to make the technologist transparent in order that their influence on the final system is reduced. It's just that I *think* that better art would result when the collaboration is between two artists, and one of them just happens to be sitting at a computer.

Another thought: The 'medium' for film directors is people. They are expressing their ideas using people, not just film. In this model, the technologist is part of the system which the 'artist' uses to create artworks. Therefore they can be experts in the 'art-producing-system' without being experts in computing. It's just that this is a complex/chaotic system!

april 7, 2004 - i choose to start this blog now

Went to doctoral assessment seminars presented by CCS people (Alastair, Greg, Roman and Dave yesterday and today. I like the fact that everyone in CCS is asking 'big questions'. Sitting in the audience I was thinking about how so many of these issues at the intersection of art and technology involve issues of control: who has it; why would an artist want it; how important is control to 'artists', etc...

For example, Greg is looking at ways to support the creation of computer based interactive artworks, or perhaps more specifically, the characteristics of programming tools which would allow artists to do what they want with computers. The 'control issue' that he faces is not new: What assumptions do you make in the design of a programming environment aimed non-technologists? Building the environment in an OO fashion a la SmallTalk, where the language is written in itself and is thus customisable at any level, seems a good approach which allows the artist to work at whichever technical level they like. Not sure if Max/MSP takes this approach. I suspect not, as the 'externals' are required to be written in C as far as I know. Certainly my personal feeling is that I'd like to be able to play around with the code in Max in a more 'programmatic' way, but I guess this reflects my background. I need to experiment with Max-style programming more...

Anyway, more issues that Greg's presentation raised for me was the distinction between art and craft. Separating the roles of 'technologist' and 'artist' implies that the artist is the one with the 'vision' and the technologist exists to simply translate these ideas into concrete form. I have a problem with this idea, as to me the craft of creation is inextricably linked to the artistic vision. That is I find it difficult to separate the idea from its execution, and to my way of thinking an artist who wishes to use the computer as his/her medium needs to understand the medium in order to express their ideas. For example, would I be considered a great artist if I had instructed da Vinci to paint the Mona Lisa? Would his art have been better if he had confined his work to being a great technologist - that is, a guy who was skilled at making marks on canvas - and had found an 'artist' to collaborate with? Is art more than an idea? Big question.. I would say it's a way of communicating a feeling, and the mechanism by which the feeling is communicated becomes part of the feeling. The medium is the message- or at least part of it - right? Oh dear..

andrew johnston blah blah

to collect & readings

• Shaw, Mildred L. G: On becoming a personal scientist : interactive computer elicitation of personal models of the world 150.2854 SHAW
• Curtis Roads: Microsound CITY CLOSED RESERVE 781.34 ROAD
• Diana Deutsch (ed): The psychology of music 1999 781.11 DEUT (ED.2)
• creativity pdf downloaded...
• Irene Deliege and Geraint Wiggins: Musical creativity: Current research in theory and practice (new- keep an eye out!)