--- theme: gaia _class: lead backgroundColor: #000000 color: #B3ACA2 --- # **Towards a universal button grid interface** ## And its open implementation Thibaud Keller [ossia.io](https://ossia.io) - [Maynooth University](https://www.maynoothuniversity.ie/) *made with [marp](https://marp.app)* ![bg right:40%](images/deluge-slider-2.jpg) ![bg](images/UGS.png) ![bg blur:2px](images/monome-gridlab.jpg) --- # An enduring practice - Thoroughly documented at LMU Munich by Beat Rossmy [[1]](https://doi.org/10.21428/92fbeb44.563bfea9), with Alexander Wiethoff [[2]](https://doi.org/10.21428/92fbeb44.6a2451e6)[[12]](https://doi.org/10.1145/3294109.3295646), Maximilian Rauh[[3]](https://doi.org/10.21428/92fbeb44.db84ecd0) and Sebastian Unger [[4]](https://doi.org/10.21428/92fbeb44.303223db). - With a dedicated course at Berkley college of Music "Grid Controller Studies" ISEL-112 [[5]](https://github.com/stretta/gridlab) - And an active community of artists and developers [[6]](https://llllllll.co/tag/grid) Due, in part, to the open source, and open-ended nature of monome's grid --- # The monome way ![w:540](images/monome-gridlab.jpg) ![h:297](images/grids.jpg) As presented in [Maker Stories [7]](https://stories.maker.co/monome-open-source-tools-for-music-makers). --- # Origins of backlit button grids ![bg vertical right:40% 70%](images/202edited.png) ![bg 70%](images/sp808_gal.jpg) 1998 - SP series (202 & 808) Performance samplers The grid allows: - Sampling and triggering - Copy and paste - Step recording --- ![bg 67%](images/grid_releases.svg) --- # Notable instruments and sequencers - 2007 - Tenori-on, Yamaha - 2014 - BeatStep, Arturia Circadian Rhythm, Tip Top audio (eurorack) - 2015 - Circuit, Novation Pocket operators, Teenage Engineering (hand held) - 2016 - Deluge, Synthstrom (open source) - 2017 - Seq, Polyend (wide) - 2019 - Zoia, Empress (nodal patcher) - 2023 - Push 3, Ableton (export to and from a PC) --- # Notable controllers - 2006 - 40h, monome (open source) - 2009 - launchpad, Novation - 2013 - Push, Ableton - 2014 - Untztrument, Adafruit (open source) - 2015 - Linstrument, Roger Lynn (continuous surface) - 2019 - Grid, Intech Studio (modular) - 2020 - Mine S, Special Waves (modular) --- # Problem - A majority of closed source, proprietary hardware. - Either a blank slate (monome), or overly standardized practices - Software specific libraries, with divergent approaches - Many overlapping features --- # Limitation ### Menus! The case of the Yamaha FS1R [[8]](https://fs1r.skerjanc.de/) and the dread of "menu diving" against [Pamela's PRO Workout](https://busycircuits.com/alm034/) ![bg right:60% 110%](images/controller-b-fur-yamaha-fs1r-730x548.jpg) ![bg 26%](images/pamela.jpg) --- # Button grids can be: - Keyboards / drum pads - Step sequencers - "Clip" launchers (Ardour (Ableton)) - Arrangement views (Deluge) - Routing matrixes (Deluge) - Rudimentary waveform croppers (Deluge) - Text displays (Launchapd) - Basic spin boxes (Circuit) - Nodal patchers (Zoia) ... --- # Tangible timelines ![w:540](images/UGS.png) ![w:540](images/superbrain.jpg) Samuel J. Hunt's [UGS](https://iv.nboeck.de/watch?v=KHFJPfa38wY) [[9]](https://doi.org/10.1145/3411109.3411122) & Beat Rossmy's [SuperBrain](https://vimeo.com/546873594) --- # Challenges - Compatibility: supporting a wide selection of hardware devices - Performance: low compute cost & optimal responsiveness - Extensibility: ease of adding more devices - Low resolution: reducing graphical elements to their bare minimum - Portability: ease of integration in software - Modularity: option to aggregate devices --- # Approach - Starting with the Launchpad Pro: common & full-featured - Implement a minimalist widget system - Porting [score](https://ossia.io)'s user interface to a grid - Drawing from the [remote control plugin](https://ossia.io/score-docs/in-depth/remote.html) - Search for Intuitive mappings of physical actions to software functionalities - Aim for full integration of the hardware device, allowing complete use of score without a screen, mouse or keyboard --- # Beyond ![bg vertical right:70% 90%](images/touchgrid.png) ![bg 90%](images/kinephone.png) Beat Rossmy's [TouchGrid](https://vimeo.com/387381202) [[4]](https://nime.pubpub.org/pub/touchgrid/release/1) & MIT labs [Kinéphone [10]](https://doi.org/10.5281/zenodo.1176145)/[SoundFORMS [11]](https://doi.org/10.1145/2851581.2892414) --- # Honorable mention ![bg vertical right:40% 90%](images/impulse_curve.jpg) ![bg 90%](images/comb.png) Eli Fieldsteel's [Impulse Curve](https://inv.nadeko.net/watch?v=6iF5FZ9DAY4&list=PLLnA8B3ZULyHtPZUaIBUtMy3xtxq_uC_I&index=8)[[14]](https://www.icfo.eu/event/3292) & Beat Rossmy's [COMB](https://vimeo.com/231299236)[[12]](https://doi.org/10.1145/3294109.3295646) --- # Bibliography [[1]](https://doi.org/10.21428/92fbeb44.563bfea9) Rossmy, B. “Buttons, Sliders, and Keys – A Survey on Musical Grid Interface Standards.” In International Conference on New Interfaces for Musical Expression, 2022. [[2]](https://doi.org/10.21428/92fbeb44.6a2451e6) Rossmy, B., & Wiethoff, A. “Musical Grid Interfaces: Past, Present, and Future Directions.” In International Conference on New Interfaces for Musical Expression, 2021. [[3]](https://doi.org/10.21428/92fbeb44.db84ecd0) Rossmy, B., Rauh, M., & Wiethoff, A. “Towards User Interface Guidelines for Musical Grid Interfaces.” In International Conference on New Interfaces for Musical Expression, 2022. --- [[4]](https://doi.org/10.21428/92fbeb44.db84ecd0) Rossmy, B., Unger, S., & Wiethoff, A. “TouchGrid – Combining Touch Interaction with Musical Grid Interfaces.” In International Conference on New Interfaces for Musical Expression, 2021. [[5]](https://github.com/stretta/gridlab) Matthew Davidson, aka stretta, gridalb patches repository, June 21, 2024. https://github.com/stretta/gridlab. [[6]](https://llllllll.co/tag/grid) Lines “Latest Grid Topics”. The monome comunity forum. https://llllllll.co/tag/grid. [[7]](https://stories.maker.co/monome-open-source-tools-for-music-makers) Maker Stories. “Monome / Open Source Tools for Music Makers.” https://stories.maker.co/monome-open-source-tools-for-music-makers. --- [[8]](https://fs1r.skerjanc.de/) Robert Skerjanc, “HARDWARE-CONTROLLER FOR YAMAHA FS1R,” August 26, 2023. https://fs1r.skerjanc.de/. [[9]](https://doi.org/10.1145/3411109.3411122) Hunt, Samuel J. “Exploring Polyrhythms, Polymeters, and Polytempi with the Universal Grid Sequencer Framework.” In Proceedings of the 15th International Audio Mostly Conference, 101–6. AM ’20. New York, NY, USA: Association for Computing Machinery, 2020. --- [[10]](https://doi.org/10.5281/zenodo.1176145) Xiao, Xiao, Donald Derek Haddad, Thomas Sanchez, Akito van Troyer, Rébecca Kleinberger, Penny Webb, Joe Paradiso, Tod Machover, and Hiroshi Ishii. “Kin\’{e}phone: Exploring the Musical Potential of an Actuated Pin-Based Shape Display.” Proceedings of the International Conference on New Interfaces for Musical Expression. June 1, 2016. [[11]](https://doi.org/10.1145/2851581.2892414) Colter, Aubrey, Patlapa Davivongsa, Donald Derek Haddad, Halla Moore, Brian Tice, and Hiroshi Ishii. “SoundFORMS: Manipulating Sound Through Touch.” In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 2425–30. CHI EA ’16. New York, NY, USA: Association for Computing Machinery, 2016. --- [[12]](https://doi.org/10.1145/3294109.3295646) Rossmy, Beat, and Alexander Wiethoff. “COMB -- Shape as a Meaningful Element of Interaction.” In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction, 287–95. TEI ’19. New York, NY, USA: Association for Computing Machinery, 2019. [[13]](https://www.icfo.eu/event/3292)Fieldsteel, Eli. “LightMatrix: A Light-Sensitive Musical Control Interface.” QUANTUM SOUNDS SYMPOSIUM - ICFO (blog). Accessed November 24, 2024.