As a resident at DBRS Innovation Labs, we worked on a Virtual Reality experience to explain one approach to machine learning. Mathematical descriptions of neural networks are precise but abstract. We wanted to give a picture of its internal workings, so that visitors could walk away with an intuitive understanding of how learning networks operate.
The project works with a convolutional neural network called LeNet, modeled after the visual cortex of a cat. Users are able to draw numbers with their hands, tracked by a Leap Motion sensor, and watch the algorithm analyze their writing in real time. Users can view the neural network from different angles and change the view to see all of the network's layers at once or zoom in on a specific layer.
Rhythm Necklaces are circular representations of repeating patterns. We first learned about them while exploring new ways to compose music. It turns out that the geometric properties of these objects—their balance and symmetry—affect not only their shape but also the way they sound. What if there was a way to compose music by playing with shapes?
Inspired by that possibility, we started a collaboration with artist and designer Meara O’Reilly. We built an iOS app that offers a new way to compose, shaping music by rotating, flipping and stretching rather than placing musical notes. Press: Wired, The Creator’s Project, Creative Applications.
Wikisonnet is an algorithmic poem generator. It grew out of our interest in computational writing, and an experiment to see if we could write poems automatically using English language Wikipedia. The algorithm pulls together lines from Wikipedia in iambic pentameter and arranges them into Elizabethan sonnets. Wikisonnet appeared at the 2016 xCoAx Conference in Bergamo, Italy.
StaRt is an iOS app built by students at the Byun Speech Lab at NYU. Working with Dr. Tara McAllister Byun, their goal was a colorful, friendly speech therapy app that kids would enjoy using.
As the project started to reach a certain size, Dr. Byun approached us to take the students’ initial work and to port it to a new architecture that would make the app more scalable and maintainable. We took the original code, which was written for iOS only, and ported it to the cross platform Ionic framework. Additionally, we separated the interface to the app from the audio backend that processes and analyzes speech, so that students with different skill sets could work on the app independently.
Max/MSP is an indispensable tool for creating interactive, audio/visual applications. We’ve been avid Max users for as long as we’ve been programming, so for us it was an amazing opportunity to work on Mira, an iPad controller app for Max/MSP.
Mira is a tool for multimedia artists to control Max/MSP while performing. It was designed to be simple to use, while establishing a stable, low-latency connection to the Max/MSP environment. In 2013, We presented Mira at the New Interfaces for Musical Expression conference, as well as Code Control Leicester. Press: Create Digital Music
Girlfriends is a creative agency based in Brooklyn.
In the past we've worked for several creative agencies, with clients including Nike, Chanel, Cycling '74, Pioneer Works, Mom + Pop, NYU, UC Berkeley Computational Science, and The Wall Street Journal.