253 lines
9 KiB
Markdown
253 lines
9 KiB
Markdown
---
|
||
theme: gaia
|
||
_class: lead
|
||
<!-- paginate: true -->
|
||
backgroundColor: #000000
|
||
color: #B3ACA2
|
||
---
|
||
|
||
# **Towards a universal button grid interface**
|
||
## And its open implementation
|
||
Thibaud Keller
|
||
[ossia.io](https://ossia.io) - [Maynooth University](https://www.maynoothuniversity.ie/)
|
||
*made with [marp](https://marp.app)*
|
||
|
||
|
||
|
||
|
||
|
||
---
|
||
|
||
# An enduring practice
|
||
|
||
- Thoroughly documented at LMU Munich by Beat Rossmy [[1]](https://doi.org/10.21428/92fbeb44.563bfea9),
|
||
with Alexander Wiethoff [[2]](https://doi.org/10.21428/92fbeb44.6a2451e6)[[12]](https://doi.org/10.1145/3294109.3295646), Maximilian Rauh[[3]](https://doi.org/10.21428/92fbeb44.db84ecd0)
|
||
and Sebastian Unger [[4]](https://doi.org/10.21428/92fbeb44.303223db).
|
||
|
||
- With a dedicated course at Berkley college of Music
|
||
"Grid Controller Studies" ISEL-112 [[5]](https://github.com/stretta/gridlab)
|
||
|
||
- And an active community of artists and developers [[6]](https://llllllll.co/tag/grid)
|
||
Due, in part, to the open source, and open-ended nature of monome's grid
|
||
|
||
---
|
||
|
||
# The monome way
|
||
|
||
 
|
||
As presented in [Maker Stories [7]](https://stories.maker.co/monome-open-source-tools-for-music-makers).
|
||
|
||
<!--
|
||
* Demo of the MLR max patch
|
||
and a sonified Conway's game of life
|
||
|
||
* Made in America
|
||
-->
|
||
|
||
---
|
||
# Origins of backlit button grids
|
||
|
||
|
||
|
||
|
||
1998 - SP series (202 & 808)
|
||
Performance samplers
|
||
|
||
The grid allows:
|
||
- Sampling and triggering
|
||
- Copy and paste
|
||
- Step recording
|
||
|
||
---
|
||
|
||
<!--
|
||
_backgroundColor: #ffffff
|
||
_header: 'Annual grid device releases ([data set](https://gothub.ducks.party/BeatRossmy/Musical-Grid-Interfaces)), separated in five product categories'
|
||
_footer: 'Figure 2 in Rosmy, B. & Wiethoff, A. “Musical Grid Interfaces: Past, Present, and Future Directions.” 2021[[2]](https://doi.org/10.21428/92fbeb44.6a2451e6)'
|
||
-->
|
||
|
||
|
||
---
|
||
|
||
# Notable instruments and sequencers
|
||
|
||
- 2007 - Tenori-on, Yamaha
|
||
- 2014 - BeatStep, Arturia
|
||
Circadian Rhythm, Tip Top audio (eurorack)
|
||
- 2015 - Circuit, Novation
|
||
Pocket operators, Teenage Engineering (hand held)
|
||
- 2016 - Deluge, Synthstrom (open source)
|
||
- 2017 - Seq, Polyend (wide)
|
||
- 2019 - Zoia, Empress (nodal patcher)
|
||
- 2023 - Push 3, Ableton (export to and from a PC)
|
||
|
||
---
|
||
|
||
# Notable controllers
|
||
|
||
- 2006 - 40h, monome (open source)
|
||
- 2009 - launchpad, Novation
|
||
- 2013 - Push, Ableton
|
||
- 2014 - Untztrument, Adafruit (open source)
|
||
- 2015 - Linstrument, Roger Lynn (continuous surface)
|
||
- 2019 - Grid, Intech Studio (modular)
|
||
- 2020 - Mine S, Special Waves (modular)
|
||
|
||
<!--
|
||
An alternative open source firmware for the Novation Launchpad Pro
|
||
|
||
Interesting, but Ideally, a library should work with any controller out of the box
|
||
-->
|
||
|
||
---
|
||
|
||
# Problem
|
||
|
||
- A majority of closed source, proprietary hardware.
|
||
- Either a blank slate (monome), or overly standardized practices
|
||
- Software specific libraries, with divergent approaches
|
||
- Many overlapping features
|
||
|
||
---
|
||
# Limitation
|
||
|
||
### Menus!
|
||
|
||
The case of the
|
||
Yamaha FS1R [[8]](https://fs1r.skerjanc.de/)
|
||
and the dread of
|
||
"menu diving"
|
||
against
|
||
[Pamela's PRO Workout](https://busycircuits.com/alm034/)
|
||
|
||
|
||
|
||
|
||
---
|
||
|
||
# Button grids can be:
|
||
|
||
- Keyboards / drum pads
|
||
- Step sequencers
|
||
- "Clip" launchers (Ardour (Ableton))
|
||
- Arrangement views (Deluge)
|
||
- Routing matrixes (Deluge)
|
||
- Rudimentary waveform croppers (Deluge)
|
||
- Text displays (Launchapd)
|
||
- Basic spin boxes (Circuit)
|
||
- Nodal patchers (Zoia) ...
|
||
|
||
---
|
||
|
||
# Tangible timelines
|
||
|
||
 
|
||
Samuel J. Hunt's [UGS](https://iv.nboeck.de/watch?v=KHFJPfa38wY) [[9]](https://doi.org/10.1145/3411109.3411122) & Beat Rossmy's [SuperBrain](https://vimeo.com/546873594)
|
||
|
||
---
|
||
|
||
# Challenges
|
||
|
||
- Compatibility: supporting a wide selection of hardware devices
|
||
- Performance: low compute cost & optimal responsiveness
|
||
- Extensibility: ease of adding more devices
|
||
- Low resolution: reducing graphical elements to their bare minimum
|
||
- Portability: ease of integration in software
|
||
- Modularity: option to aggregate devices
|
||
|
||
---
|
||
|
||
# Approach
|
||
|
||
- Starting with the Launchpad Pro: common & full-featured
|
||
- Implement a minimalist widget system
|
||
- Porting [score](https://ossia.io)'s user interface to a grid
|
||
- Drawing from the [remote control plugin](https://ossia.io/score-docs/in-depth/remote.html)
|
||
- Search for Intuitive mappings of physical actions to software functionalities
|
||
- Aim for full integration of the hardware device, allowing complete use of score without a screen, mouse or keyboard
|
||
|
||
<!--
|
||
score contains:
|
||
Independent timelines,
|
||
nodal patchers,
|
||
waveform displays,
|
||
and widgets of all kind
|
||
|
||
Heavily rely on WIMP interactions
|
||
-->
|
||
|
||
---
|
||
|
||
# Beyond
|
||
|
||
|
||
|
||
Beat Rossmy's [TouchGrid](https://vimeo.com/387381202) [[4]](https://nime.pubpub.org/pub/touchgrid/release/1) & MIT labs [Kinéphone [10]](https://doi.org/10.5281/zenodo.1176145)/[SoundFORMS [11]](https://doi.org/10.1145/2851581.2892414)
|
||
|
||
<!--
|
||
edge and basal interactions
|
||
|
||
Accessibility: convert light lo cell height
|
||
or vibration
|
||
|
||
small screen in each cell
|
||
or dot matrix
|
||
or 14 segment displays
|
||
|
||
displaying characters requires at least 3x5 cell fonts
|
||
https://fontstruct.com/fontstructions/show/1371235/3x5-dot-matrix-1
|
||
-->
|
||
|
||
---
|
||
|
||
# Honorable mention
|
||
|
||
|
||
|
||
Eli Fieldsteel's
|
||
[Impulse Curve](https://inv.nadeko.net/watch?v=6iF5FZ9DAY4&list=PLLnA8B3ZULyHtPZUaIBUtMy3xtxq_uC_I&index=8)[[14]](https://www.icfo.eu/event/3292)
|
||
& Beat Rossmy's
|
||
[COMB](https://vimeo.com/231299236)[[12]](https://doi.org/10.1145/3294109.3295646)
|
||
|
||
<!--
|
||
Impulse curve filmed performance
|
||
https://inv.nadeko.net/watch?v=k8N0iMYd9H8&t=0
|
||
-->
|
||
|
||
---
|
||
|
||
# Bibliography
|
||
|
||
[[1]](https://doi.org/10.21428/92fbeb44.563bfea9) Rossmy, B. “Buttons, Sliders, and Keys – A Survey on Musical Grid Interface Standards.” In International Conference on New Interfaces for Musical Expression, 2022.
|
||
|
||
[[2]](https://doi.org/10.21428/92fbeb44.6a2451e6) Rossmy, B., & Wiethoff, A. “Musical Grid Interfaces: Past, Present, and Future Directions.” In International Conference on New Interfaces for Musical Expression, 2021.
|
||
|
||
[[3]](https://doi.org/10.21428/92fbeb44.db84ecd0) Rossmy, B., Rauh, M., & Wiethoff, A. “Towards User Interface Guidelines for Musical Grid Interfaces.” In International Conference on New Interfaces for Musical Expression, 2022.
|
||
|
||
---
|
||
|
||
[[4]](https://doi.org/10.21428/92fbeb44.db84ecd0) Rossmy, B., Unger, S., & Wiethoff, A. “TouchGrid – Combining Touch Interaction with Musical Grid Interfaces.” In International Conference on New Interfaces for Musical Expression, 2021.
|
||
|
||
[[5]](https://github.com/stretta/gridlab) Matthew Davidson, aka stretta, gridalb patches repository, June 21, 2024. https://github.com/stretta/gridlab.
|
||
|
||
[[6]](https://llllllll.co/tag/grid) Lines “Latest Grid Topics”. The monome comunity forum. https://llllllll.co/tag/grid.
|
||
|
||
[[7]](https://stories.maker.co/monome-open-source-tools-for-music-makers) Maker Stories. “Monome / Open Source Tools for Music Makers.” https://stories.maker.co/monome-open-source-tools-for-music-makers.
|
||
|
||
---
|
||
|
||
[[8]](https://fs1r.skerjanc.de/) Robert Skerjanc, “HARDWARE-CONTROLLER FOR YAMAHA FS1R,” August 26, 2023. https://fs1r.skerjanc.de/.
|
||
|
||
[[9]](https://doi.org/10.1145/3411109.3411122) Hunt, Samuel J. “Exploring Polyrhythms, Polymeters, and Polytempi with the Universal Grid Sequencer Framework.” In Proceedings of the 15th International Audio Mostly Conference, 101–6. AM ’20. New York, NY, USA: Association for Computing Machinery, 2020.
|
||
|
||
---
|
||
|
||
[[10]](https://doi.org/10.5281/zenodo.1176145) Xiao, Xiao, Donald Derek Haddad, Thomas Sanchez, Akito van Troyer, Rébecca Kleinberger, Penny Webb, Joe Paradiso, Tod Machover, and Hiroshi Ishii. “Kin\’{e}phone: Exploring the Musical Potential of an Actuated Pin-Based Shape Display.” Proceedings of the International Conference on New Interfaces for Musical Expression. June 1, 2016.
|
||
|
||
[[11]](https://doi.org/10.1145/2851581.2892414) Colter, Aubrey, Patlapa Davivongsa, Donald Derek Haddad, Halla Moore, Brian Tice, and Hiroshi Ishii. “SoundFORMS: Manipulating Sound Through Touch.” In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 2425–30. CHI EA ’16. New York, NY, USA: Association for Computing Machinery, 2016.
|
||
|
||
---
|
||
|
||
[[12]](https://doi.org/10.1145/3294109.3295646) Rossmy, Beat, and Alexander Wiethoff. “COMB -- Shape as a Meaningful Element of Interaction.” In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction, 287–95. TEI ’19. New York, NY, USA: Association for Computing Machinery, 2019.
|
||
|
||
[[13]](https://www.icfo.eu/event/3292)Fieldsteel, Eli. “LightMatrix: A Light-Sensitive Musical Control Interface.” QUANTUM SOUNDS SYMPOSIUM - ICFO (blog). Accessed November 24, 2024.
|