Patchies - Bucket List

These are the features for Patchies I have implemented so far, and going to implement next. It's a big list!

  • Video Chaining
  • Strudel.cc Integration
  • P' Giang: JS ต่อ LLM แล้วพ่น text + image แล้วส่งเป็น pipeline เข้า image feed
  • Background Output Node
  • Butterchurn
  • Add object palette -- quick add object at cursor
  • LLM PromptDJ MIDI Gemini
  • LLM Text Generation
  • P' Giang: ดูแล้วเราอาจจะทำ output node เพื่อเอาไปออกอีกหน้าจอได้ แล้วเหมือนพวกนี้เป็น preview รอวันส่งออกไป แต่ดูแล้วเหมือนน่าจะทำได้เลย แบบว่า extend จอแล้วขยายหน้าต่างเอา
  • P' Henry: Fullscreen (Entire Program)
  • Saving and loading patches (serialization and deserialization)
  • Command Palette
  • Fix the video chaining bugs
  • Fix output node chaining bugs
  • SwissGL
  • Refactor common node objects
  • Text-based objects and parameters (like in PureData and Max/MSP, useful for audio nodes)
  • Native Slider object
  • Audio Node Graph with Web Audio API, PureData style
  • Text objects and parameters
  • LFO
  • ADSR Message
  • ADSR Node
  • FM Synthesis
  • Audio Analysis for P5.js
  • Audio Analysis for Hydra.js
  • Audio Analysis for GLSL
  • Python on Browser with Pyodide
  • Send simple messages
  • Moving Python to run on web worker
  • AM Synthesis
  • Microphone Input
  • Markdown block
  • LPF / HPF / BPF
  • SetPortCount for P5
  • SetPortCount for Hydra
  • Custom expressions (Expr)
  • Custom expressions in Audio Worklet (Expr~)
  • Audio Sampler
    • Drag in audio into canvas
    • Audio Recorder into sample
  • Drag in image file (image object)
  • Drag in text file (markdown)
  • Loadbang (Bang on load)
  • Metro
  • Basic sampler loop by defining 2 points
  • Color code edges
    • Audio
    • Video
    • Message (Parameter)
  • Networked Messages: Send & Receive across computer
  • Fix video inlet bug
  • FFT returns a FFTAnalysis that contains useful computations, liek getting energy and rms.
  • Screen Sharing Node
  • Webcam Node
  • Control Rate Clock (Metro)
  • Run custom JS code in AudioWorklet (dsp~)
  • Run custom Tone.JS code (tone~)
  • Time t in expr~
  • Spigot control object
  • Toggle Button
  • Vertical Slider (for mixers)
  • Label block (looks exactly like message, but clicking does nothing + no ports + no outline)
  • Link block -- go to link. Show a warning first.
  • Sharing reusable JavaScript code
  • Bug: JS console maxWidth exceeds content width
  • Limit value inlet count to max 20
  • setRunOnMount bug when JS has dependencies
  • Visual Assembly Canvas
    • Stack Machine block + ASM Interpreter
    • Bring over the visual blocks + integrate with standard messaging
  • Allow visual assembly canvas to run on workers
  • Pocketbase - CRUD Patch
  • Preset: Simple Plotter
    • may want to expand this to be dynamic to data shape
  • expr: always expose at least one expr inlet to send bang
    • as recommended by P' kijjazz
  • asm: Shift+Enter to autorun
  • asm: pause/play should detect if stuck in 'ready' state (or halted), and allow playing again.
  • asm: reset playback state when machine status is halted or ready
  • asm: disable step button when machine is halted
  • ai.img: fix code editor width is super small
  • autosave: only autosave when tab is active and focused. prevents other browser tabs from overriding autosaves.
  • chuck: make "replace shred" the first icon button, and make "add shred" button the second icon. add shred should use an add icon, not a play icon.
  • asm: memory cell block to store temporary or persistent data
  • snapshot~: snapshot audio signal values
  • meter~:
    • meter loudness
    • show number when peaked
    • use dBFS
    • เพราะถ้าจะ peak แตก เห็นเลยว่ากำลังแตะค่าสูงสุดแถวไหน
    • บาง meter จะมีแบบ ถ้าเกิน 0 dBFS ไฟจะแดงติดค้าง ต้องไปกดที่ไฟแดงถึงจะ reset ไฟแดง คือเอาไว้เตือนว่าอะไรแตกง่าย คุมไหวไหม
  • asm: memory cell block should autofill zeroes on write, don't leave null at unknown locations.
  • split~ n: ChannelSplitterNode
  • merge~ n: ChannelMergerNode
  • keyboard: bind to keydown and keyup event
  • volume control slider
  • elementary audio node (elem~)
  • csound basic integration (csound~)
  • js: resize object automatically for outlets
  • p5: wrong index of message outlets (off-by-one)
  • sampler: repeated playback on sampler causes sample to be shortened permanently
  • better sampler
    • set start and end points to play sample
    • looping (taking into account start and end)
    • show waveform so start/end is easier
    • playback rate and detune
    • bugfix: waveform gets squished vertically everytime when start and end is adjusted
    • bugfix: old waveform stays even when new sample is recorded
    • recommended by P' Thai
  • p5: fix lazy loading causing P5 to have no width/height
    • regex match code against createCanvas(x, y), then use that to pre-fill the expected width/height
  • welcome screen
    • patch examples
    • link to documentation / getting started
    • browse saved patches
    • object browser / object help
  • patch examples / showcase gallery
    • refs: tldraw computer, tweakable.org
  • drag to select multiple nodes
    • move multiple nodes at once
    • copy and paste multiple selected nodes and edges at once
  • save/load: avoid visiting new patch (e.g. ?id=...) overriding your non-saved autosave
    • maybe autosaves should have named slots?
    • default -> no id
    • id=... -> playing from a fork
    • named patch -> save to the patch name
  • CTRL+S to save patches
    • if it's Unsaved, prompt them to save
  • in-app examples and snippets for each node
    • like Max/MSP's in-app help object
  • documentation website
  • sharing links: visiting a patch should not erase the id url parameter from the url, makes sharing harder
  • undo and redo stack
    • recommended by P' Thai
  • csound~: more integration
    • proper message handling
    • supporting audio inputs
    • dynamic inlets/outlets
  • fix memory leaks
    • gets up to 200GB now in the polyphony patch 😰
  • offline support
    • patcher should not always require internet. downloaded assets (e.g. webchuck, esm) shall be cached in web workers
    • option to "load every lazy bundle" to ensure offline usage does not miss any features (e.g. CodeEditor still not loaded), and show loading progress.
  • virtual filesystem
    • Providers: local, url (store as url), cloud
      • either online, offline locateable with filesystem api
      • offline locating is similar to Max MSP's search paths
      • make sure to define extensible interfaces for file uploads
    • Each object only store vfs path, it must not store the file object. Use the VFS API to load.
    • elem~: integrate with Elementary Audio's virtual filesystem
    • csound~: integrate with CSound's virtual filesystem
    • chuck~: integrate with Chuck's createDirectory, createFile, loadFile
    • p5 loadImage (like the P5.js editor)
    • Recorded samples in sampler~
    • persist images (img), sound (soundfile~), video (video)
    • for public use, maybe set a really tiny max file size, and we would also need uploads
  • refactor the super long files
    • refactor system singleton classes - god classes
      • AudioSystem (1200 lines)
      • fboRenderer (1000 lines)
      • GLSystem (360 lines), also should rename to VideoSystem
      • AssemblySystem (400 lines)
      • MIDISystem (300 lines)
      • JSRunner (350 lines)
    • refactor huge svelte nodes - files over 200 lines
  • AI assistant/agent
    • simple chat
    • ref: talk to the hand. youtube sigchi, sigchi program
    • customizable prompts / build your own agent
    • builds on top of serializable patcher and headless patching, and expose operations as LLM tool calls
    • agent autonomously perform tasks on the patcher, get approval and input from the user
      • TODO, Code Approval, Tool Call
    • using Vercel's AI SDK & tool calling
    • goal: co-creation, collaboration, constructivist learning
  • headless patcher api
    • use patchies as a framework for creating your own experiences
    • to prepare for nesting patches
    • differentiate Svelte component mount/unmount lifecycle from headless patch mount/unmount
    • run only the services
  • sub-patching and nested patches
    • import patches with abstractions (drag in JSON file)
  • asm: critical optimizations to speed things up a lot~
    • don't always call syncMachine
    • only query data that is strictly needed.
      • machineExists should NOT use inspect_machine thats dumb
    • batch the check operations
  • expr~: multiple signal inputs inside expr~ to compute
    • e.g. using s0 + s1 should automatically expose two blue signal inlets
  • dsp~: support usage for multiple audio inputs
    • right now, inputs always refer to one connector.
  • sharing it with the world~
    • create discord channel
    • bad apple with asm block
    • shader graph
    • hacker news
  • workerize: allow js to run on workers
  • deworkerize: allow canvas to run on main thread
    • for tighter fft reactivity
  • toggle: allow numeric outputs
    • send a message {type: 'number'} for toggle to emit 0 and 1. {type: 'boolean'} to emit booleans
  • virtual console: add console to all JS blocks other than js for easier logging view
    • error handling: redirect all errors in all objects into the virtual console.
      • should output P5, GLSL and canvas errors correctly
    • make console window resizeable
    • js: better message console
      • make it colored?
      • make it interactive? (e.g. browsing JSON objects?)
  • onboarding: add popup window (like hydra) for onboarding
      • sample patcher with explanations
    • see Strudel and Hydra for ideas
  • sheets: add data sheet like google sheet
  • slider: resizeable slider (make vertical sliders taller, make horizontal sliders wider)
  • mouse tracking for P5, canvas, GLSL and hydra
    • global mouse vs local mouse (preview + canvas position remapped)
    • js: allow setting mouse mode to screen vs preview: setMouseScope('screen') vs setMouseScope('preview')
  • docs for JS imports (JSRunner): P5, JS, and canvas
    • add the "canvas" to README
  • timelines, automation scheduling and sequencing
    • as recommended by P' kijjazz
    • create separate automation tracks/timelines and run automation concurrently
    • ability to visualize and define points
    • ref: juggle and hide by duck unit robot automation
  • sample-accurate scheduling algorithm? 
    	nextStep = Math.ceil(audioContext.time / period)
	scheduledTime = nextStep * period
	run faster than period ->
		never send duplicate nextStep
  • uxntal, varvara and orca
  • user presets / snippets library
    • user can save an object as a new preset e.g. "mix" preset
    • not limited to our provided preset
    • should be easy to turn any object into a preset
  • codemirror autocompletions
    • JS
    • GLSL
    • Hydra
    • Strudel
    • Assembly
  • wireless send/receive
    • message (message bus)
    • audio (audio bus)
      • e.g. send reverb to mix
      • dry/wet
      • recommended by P' Kij
    • video (video bus)
  • drum sequencer
    • define bpm as global clock?
  • sfz sample support
    • recommended by P' kij
  • fexpr~ (expr~ with circular ring buffer)
    • recommended by P' kij
  • grouping block - for providing sections
    • recommended by Chun
  • color picker object
  • numeric slider in code editor
    • glsl
    • js
  • color picker in code editor
    • glsl
    • js
  • x/y pad in code editor
    • glsl
    • js
  • x-y pad
  • graph function / curve editor
  • pixel art preset for use with asm as video output
  • auto-refresh P5.js
    • like in p5 website editor, no need to hit run
  • embedded patchies
    • via web components / custom element api
    • as a native svelte component
    • via iframes
    • via headless Patcher class
  • using stack.gl or some libraries/utilities for glsl
  • object: show sliders or numeric input on text object parameter click
    • Drag up and down to change number (like PureData)
    • If min/max defined -> show slider range
    • If no min/max -> show just text input
  • message: allow dragging up/down if message is a pure number, like in puredata
    • no need for sliders
  • knobs: like sliders but look nicer in some cases
  • easier midi parameter mapping
  • real-time live collaboration
  • synchronized clocks for real-time play together
  • web audio modules 2 integration
    • recommended by P' Thai and P' Kij
    • can rust faust
  • preset: number display
  • golang server
  • OSC (OpenSourceControl) over WebSocket proxy server
    • recommended by P' Henri
  • SuperCollider integration
  • LLM Video Generation e.g. stream diffusion
  • user-made objects with js
    • custom user-made text-based objects using the ObjectNode framework
    • create user-made custom visual objects. maybe svelte or js.
  • uiua interpreter
  • 3D rendering with three.js or other libraries
  • granular synthesis
  • dynamically sized video objects
    • right now you can only have one size
  • rust webassembly plugin system
  • add value inlets $1 to $9 to some blocks?
    • easier than using recv for simple cases
  • maximize a node preview temporarily to be on-screen
    • right now you need to connect to bg.out manually
  • virtual players (like renick bell?)
Back to Home