Patchies - Bucket List

These are the features for Patchies I have implemented so far, and going to implement next. It's a big list!

  • Video Chaining
  • Strudel.cc Integration
  • JS ต่อ LLM แล้วพ่น text + image แล้วส่งเป็น pipeline เข้า image feed
  • Background Output Node
  • Butterchurn
  • Add object palette -- quick add object at cursor
  • LLM PromptDJ MIDI Gemini
  • LLM Text Generation (recommended by giang_ws)
  • P' Giang: ดูแล้วเราอาจจะทำ output node เพื่อเอาไปออกอีกหน้าจอได้ แล้วเหมือนพวกนี้เป็น preview รอวันส่งออกไป แต่ดูแล้วเหมือนน่าจะทำได้เลย แบบว่า extend จอแล้วขยายหน้าต่างเอา
  • P' Henry: Fullscreen (Entire Program)
  • Saving and loading patches (serialization and deserialization)
  • Command Palette
  • Fix the video chaining bugs
  • Fix output node chaining bugs
  • SwissGL
  • Refactor common node objects
  • Text-based objects and parameters (like in PureData and Max/MSP, useful for audio nodes)
  • Native Slider object
  • Audio Node Graph with Web Audio API, PureData style
  • Text objects and parameters
  • LFO
  • ADSR Message
  • ADSR Node
  • FM Synthesis
  • Audio Analysis for P5.js
  • Audio Analysis for Hydra.js
  • Audio Analysis for GLSL
  • Python on Browser with Pyodide
  • Send simple messages
  • Moving Python to run on web worker
  • AM Synthesis
  • Microphone Input
  • Markdown block
  • LPF / HPF / BPF
  • SetPortCount for P5
  • SetPortCount for Hydra
  • Custom expressions (Expr)
  • Custom expressions in Audio Worklet (Expr~)
  • Audio Sampler
    • Drag in audio into canvas
    • Audio Recorder into sample
  • Drag in image file (image object)
  • Drag in text file (markdown)
  • Loadbang (Bang on load)
  • Metro
  • Basic sampler loop by defining 2 points
  • Color code edges
    • Audio
    • Video
    • Message (Parameter)
  • Networked Messages: Send & Receive across computer
  • Fix video inlet bug
  • FFT returns a FFTAnalysis that contains useful computations, liek getting energy and rms.
  • Screen Sharing Node
  • Webcam Node
  • Control Rate Clock (Metro)
  • Run custom JS code in AudioWorklet (dsp~)
  • Run custom Tone.JS code (tone~)
  • Time t in expr~
  • Spigot control object
  • Toggle Button
  • Vertical Slider (for mixers)
  • Label block (looks exactly like message, but clicking does nothing + no ports + no outline)
  • Link block -- go to link. Show a warning first.
  • Sharing reusable JavaScript code
  • Bug: JS console maxWidth exceeds content width
  • Limit value inlet count to max 20
  • setRunOnMount bug when JS has dependencies
  • Visual Assembly Canvas
    • Stack Machine block + ASM Interpreter
    • Bring over the visual blocks + integrate with standard messaging
  • Allow visual assembly canvas to run on workers
  • Pocketbase - CRUD Patch
  • Preset: Simple Plotter
    • may want to expand this to be dynamic to data shape
  • expr: always expose at least one expr inlet to send bang
    • as recommended by kijjaz
  • asm: Shift+Enter to autorun
  • asm: pause/play should detect if stuck in 'ready' state (or halted), and allow playing again.
  • asm: reset playback state when machine status is halted or ready
  • asm: disable step button when machine is halted
  • ai.img: fix code editor width is super small
  • autosave: only autosave when tab is active and focused. prevents other browser tabs from overriding autosaves.
  • chuck: make "replace shred" the first icon button, and make "add shred" button the second icon. add shred should use an add icon, not a play icon.
  • asm: memory cell block to store temporary or persistent data
  • snapshot~: snapshot audio signal values
  • meter~:
    • meter loudness
    • show number when peaked
    • use dBFS
    • เพราะถ้าจะ peak แตก เห็นเลยว่ากำลังแตะค่าสูงสุดแถวไหน
    • บาง meter จะมีแบบ ถ้าเกิน 0 dBFS ไฟจะแดงติดค้าง ต้องไปกดที่ไฟแดงถึงจะ reset ไฟแดง คือเอาไว้เตือนว่าอะไรแตกง่าย คุมไหวไหม
  • asm: memory cell block should autofill zeroes on write, don't leave null at unknown locations.
  • split~ n: ChannelSplitterNode
  • merge~ n: ChannelMergerNode
  • keyboard: bind to keydown and keyup event
  • bug: repeated playback on sampler causes sample to be shortened permanently
  • volume control slider
  • elementary audio node (elem~)
  • csound basic integration (csound~)
  • js: resize object automatically for outlets
  • p5: wrong index of message outlets (off-by-one)
  • better sampler
    • set start and end points to play sample
    • looping (taking into account start and end)
    • show waveform so start/end is easier
    • playback rate and detune
    • bugfix: waveform gets squished vertically everytime when start and end is adjusted
    • bugfix: old waveform stays even when new sample is recorded
    • recommended by dtinth
  • onboarding: add popup window (like hydra) for onboarding
    • see Strudel and Hydra for ideas
  • welcome screen
  • refactor AudioSystem (1200 lines)
  • orca
  • uxn, uxntal and varvara
  • create example demo patches
  • create user-made custom visual objects with canvas.
  • x-y pad preset
  • color picker preset
  • share on llllllll.co
  • codemirror autocompletions: JS
  • AI assistant: create objects
  • p5: fix lazy loading causing P5 to have no width/height. regex match code against createCanvas(x, y), then use that to pre-fill the expected width/height
  • CTRL+S to save patches: save to autosave by default
  • drag to select multiple nodes and move multiple nodes at once -> use shift + drag
  • copy and paste multiple selected nodes and edges at once
  • create a de-workerized canvas canvas to run on main thread for tighter fft reactivity
  • supersonic integration i.e. supercollider on the web
  • Curate examples for README
  • patch examples / showcase gallery like tldraw computer and tweakable.org
  • README: start with "here's what you can do with Patchies" with screenshots
  • granular synthesis (use ChucK or Tone)
  • docs for JS imports, add canvas JSRunner support to readme
  • mouse tracking for P5
  • mouse tracking for canvas.dom
  • mouse tracking for GLSL
  • mouse tracking for hydra
  • Patchies - Virtual Console
  • 3D rendering with Three.js
  • performance optimizations for when there is lots of visual nodes: async pixel reads, viewport culling, removing backdrop-blur
  • sampler: repeated playback on sampler does not play again
  • sampler: the sampler duration gets shortened when recording a short sample once, the sampler becomes broken
  • sampler: should not capture silence at start of sample
  • soundfile: convert soundfile~ to sampler~
  • netsend/netrecv: use peerjs for p2p
  • OSC (OpenSourceControl) over WebSocket proxy server
    • recommended by P' Henri and P' Peachy
    • supported by P' Thai's external call
  • Patch-local js snippet using virtual filesystem
  • Patchies - Virtual Filesystem
  • audio input/output device selection
  • option to disable audio processing in mic~
  • fix chrome rendering issue where bg out becomes black when fbo graph is updated
  • add inline open browser link from quick insert object menu (Pub's idea)
  • add presets to object browser
  • fix shorthands and alises being in object browser
  • fix default text object parameters not added thru object browser
  • change font: IBM Plex Mono
  • Set app icon
  • Patchies - User Presets Library
  • reduce clutter on bottom toolbar buttons
  • make sidebar (files and presets) mobile friendly
  • profile and fix 200GB memory leaks -- was caused by setBitmap
  • add worker object to let js code run on workers
  • Prioritize common objects in both object browser and object insert
  • support MediaPipe wasm for AI pose, gesture -- use the worker mechanism
  • Optimize webcam and video objects with MediaStreamTrackProcessor and MediaBunny
  • disable tap to connect when EASY CONNECT is off
  • marketplace - choose what objects and presets you want
  • add menu to browse saved patches
  • CTRL+S: prompt them to save patch
  • add command to set netsend/netrecv room
  • bug: visiting new patch (e.g. ?id=...) overrides your autosave
  • bug: when sharing links, visiting a patch should not erase the id url parameter from the url as it makes sharing harder
  • offline support - downloadable assets (e.g. bundle, webchuck, esm) must be cached
  • offline support - option to "load every lazy bundle" to ensure offline usage does not miss any features
  • add waveform icons to oscillator's type parameter
  • rename dac~ to out~
  • make trigger (t) a visual object with hint
  • WebGPU compute shaders
  • documentation website
  • single source of message docs/matches: inline message inlet/outlet docs + handleMessage matcher + messaging tables for docs website
  • in-app examples and snippets for each node
  • add wireless send/receive object for messages
  • add wireless send/receive for messages in JS
  • add wireless send/receive object for audio
  • add wireless send/receive object for video
  • fix text object aliases not showing up in autocomplete
  • unify netsend/netrecv object layout with vdo.send and vdo.recv
  • use to/from in JS API
  • migrate objects to use schemas for message matching and docgen
  • update asm instruction docs
  • add schema for asm.mem
  • attrib: add hydra-ts as local fork
  • attrib: add pwa-sveltekit
  • asm: smaller memory size per asm instances
  • asm: critical optimizations to speed things up
  • asm: fix bug where re-run after code change does not reset machine
  • uxn: compact layout which hides screens
  • uxn: add virtual file support for loading Uxn ROM
  • uxn: dropping real ROM files onto canvas creates Uxn node
  • uxn: disable screen devices to be headless on compact mode
  • Patchies - Patch to Prompt
  • Patchies - Patch to Webapp
  • refine patch to webapp usability
  • bug: canvas.dom cannot use noWheel/noPan/noInteract
  • bug: sidebar scroll must keep header on top when changing pages
  • bug: patch to app should be fullscreen on mobile
  • bug: patch to app should not have close button, only minimize
  • kv object for key-value store
  • kv instance for key-value store in JS
  • duplicate edges too when bulk copying
  • expr~: accept multiple audio signal inputs (s1 - s9)
  • dsp~: support multiple audio signal inputs and outputs
  • i/f: mark hot inlet as hot visually in StandardHandle
  • i/f: add docs for int and float
  • bug: add aliases to schema mapping for s/r/int/float
  • bug: sending bang on tap log($1) prints undefined
  • asm.mem docs: re-check docs on virtual memory map
  • bug: elem~ setTitle does not adjust code editor left position
  • slider: make slider resizable horizontally and vertically
  • bug: prevent saved patch overrides
  • bug: prevent saved preset overrides
  • basic undo and redo
  • refactor: cleanup FlowCanvasInner a bit
  • add post-it note
  • add security disclaimer on loading shared patches
  • undo: commit code changes to history on CodeMirror blur
  • undo: commit node data changes for non-code nodes
  • undo: commit node data changes for object node
  • webcam: add webcam camera selector
  • add circular encoder knob
  • add simple toggle switch
  • knob: only show settings when selected on mobile
  • fix: paste button never shows up after refactor
  • apply global patch lock for slider/knob/note
  • dsp~: fix broken snapshot~ due to multiple audio inputs
  • don't apply read-only if user did not load yet / cancel load
  • make read only, audio and easy connect banner mobile-responsive
  • msg should not have autocomplete as it is JSON5
  • jsrunner docs should link to all mentioned objects
  • document hot-cold inlet on message passing docs
  • asm: address coderabbit reviews
  • asm: show human-readable error rather than JSON
  • asm: read-only code editor when program is running
  • asm: improve toolbar icon and tooltip
  • fix: dialog missing background
  • demo: repair broken trigger on adsr demo
  • help: add adsr help patch
  • native worklet-based audio processing objects
  • add line~, phasor~ snapshot~, noise~, bang~
  • support default arguments on audio math operators
  • msg: support comma-separated sequential messages
  • msg: support space-delimited messages for arrays
  • msg: support object field shorthands for object messages
  • slider/knob: support slider <min> shorthand without max
  • bug/jsrunner: on message callback does not clear across runs
  • audio scope with resize, buffer size, x/y and fps settings
  • ai: dialog should take sidebar width into consideration
  • textbox: make resizeable and lockable
  • scope~: add plot type and decay
  • add samphold~, adsr~, env~, vline~, latch~
  • use single audio and text list for schema generation
  • msg: context-aware shorthand filtering
  • scope~: lissajous plot from 2 inputs
  • soundfile~: stream audio by url via VFS url provider
  • scope~: scale x independent of y in lissajous mode
  • object: apply sticky negative sign to numeric params
  • ensure we start from an empty patch when loading shared patch
  • tone~: dispose tone.js objects
  • add buffer bridge service for SharedArrayBuffer with fallback
  • firefox - use snapshots for no-SAB mode
  • tabread~ / tabwrite~ for writing audio samples
  • docs are very slow to load
  • firefox: show iframe warning when COEP is enabled
  • packs: split audio packs into multiple packs
  • packs: move low-level pack below scripting
  • fft~: normalize getEnergy into 0 - 1
  • ask for the file name when exporting a patch
  • console: show logged texts together in one wrapped line
  • presets: do not persist built-in presets to storage
  • AudioParam modulation infra for native DSP objects
  • show blue inlet on audio param inlet when signal is connected
  • modulation: phasor~, pulse~, clip~, comb~, pow~
  • pulse~: add set phase inlet
  • help: show audio parameters in blue in schema table
  • show blue ring and title for audio parameters
  • add ftom~, mtof~, sqrt~, log~, exp~, rsqrt~
  • add fexpr~ (expr~ with circular ring buffer)
  • add tabosc4~ and cos~
  • fix: set unscheduled audio parameter values
  • add vcf~, biquad~ and slop~
  • rename f and i to float and int
  • add uiua stack based language object
  • boost logger.js in quick add and object browser search
  • fix: audio parameters broken on Google Chrome
  • unify slider styles + components in object settings
  • use mute button as DSP off
  • add speech to text (ai.stt) via Gemini API
  • uiua: add run button into the code editor
  • Patchies - Global Transport Control
  • Make transport control mobile responsive
    • sample-accurate event scheduling
  • csound: support multiple csound instances
  • ai: prompt dialog should expand from top right when thinking
  • transport: networked transport sync
  • built-in drum sequencer with velocity support
  • show drum sequencer marker in timeline
  • sequencer: control messages for sequencers
  • maximize a video node preview temporarily to be on-screen
  • bug: sharing link on iOS fails to copy the link
  • function breakpoint and curve editor
  • bug: deleting a background-overridden object should unpin it
  • sampler~: download buffer content as .wav file
  • curve: allow deleting points and dragging object on mobile
  • Patchies - Sample Search Sidebar Panel
  • sample-browser: right click context > copy as strudel name
  • sample-browser: scsynth synthdefs/samples search & insert
  • allow hiding cables
  • persist bpm and time signature to patch
  • persist cable show/hide to patch
  • add unpack object to unpack n elements from array
  • unpack: last outlet for list of n items exceeding n outlets
  • sample-browser: auto preview sounds + arrow up/down
  • sample-browser: set preview volume (both auto and manual)
  • patch-to-app: show in fullscreen button
  • fix(saves): import patch via palette overrides current patch
  • fix(ai-edit): can ctrl+i to change mode
  • sample-browser: add user tags for search
  • sample-browser: insert soundfile~ on mobile
  • fix(soundfile~): convert to table does not work
  • Patchies - Float Table Visualizer
  • feat(table, sampler~): mouse wheel to zoom in/out of sample
  • sampler~: fix cannot drag within settings empty space
  • feat(sample-browser): drop sample into sampler~
  • feat(sample-browser): drop sample into table
  • table: link to VFS (e.g. from soundfile~ conversion)
  • table: add VFS unlink button
  • table: add load msg to load by VFS path or url
  • sampler~: add load msg to load by VFS path or url
  • feat(sample-browser): copy sample as url
  • feat(file-tree): multi-select with shift + click
  • feat(table, sampler~): set vfsPath on load msg
  • feat(sample-browser): add to "Samples" vfs folder
  • feat(table): write array of float into table
  • feat(table): add clear and normalize right click menu
  • signal: allow sending float messages to signal inputs
  • fix: can't reverse-drag audio param inlet into signal outlet
  • signal: sticky precision for signal float display
  • fix(hydra, swgl, three): inlet index should not contain video
  • fix(vfs): accept ogg audio files
  • Patchies - AI Chat
  • Patchies - More AI Create and Edit Modes
  • ai chat: tool call for multi-shot editing
  • ai: fork mode
  • ai: read patch metadata and read object data
  • ai: multiple chat instances
  • ai: multiple quick edit instances (now it's sequential)
  • ai: activity tray for multiple quick edits
  • ai: tool call for docs lookup
  • ai tray: hide on mobile when sidebar is open
  • ai tray: should transition to Cooking objectName... on create
  • ai: button should not go into loading mode
  • ai: show mode-dependent button colors
  • ai: upload photo into chat
  • ai: chat personas
  • ai: add KaTeX math support
  • fix: recv should be able to register multiple callbacks & wireless callbacks
  • fix: recv should be able to register multiple callbacks in render workers
  • fix: send and receive messages wirelessly in render workers
  • fix(clock): clock.every, clock.onBeat and clock.schedule callback does not emit error
  • ai: persist chat history
  • Patchies - Object Performance Profiler
  • profiler: display avg/max/p95/last/cps
  • profiler: cycle main display stats
  • profiler: display render thread profiler graph
  • profiler: profile p5, canvas.dom, three.dom, textmode.dom
  • profiler: settings for sample window, update rate, threshold
  • strudel: limit width/height of strudel editor
  • fix(ai): stop chat does not work
  • Patchies - Broken iOS Safe Areas
  • bug(ai): tool call tends to do multi create when unnecessary
  • bug(ai): edit/add tool calls do not persist in chat, so the agent builds the same thing over and over because they think it is not built yet
  • bug(ai): agent messages are lost / not persisted sometimes
  • Canvas Objects: three dots menu should have re-run
  • slider touch target on mobile is tiny
  • ai: ask/create/edit chat modes
  • Patchies - Server Persistence
  • Patchies - Dynamic Object Settings API
  • Patchies - Dynamic AudioParam API
  • Patchies - Embodied Agents
  • sampler~: sync long media file playback to transport
  • Patchies - Fix AI Edge Generation
  • Patchies - LoD Preview Optimization
  • sampler~: play at target audioContext time
    • start(when, offset, duration) of AudioBufferSourceNode
    • {type: 'play', when?, offset?}
    • offset in seconds
  • clock: send
    • figure out wtf is wrong with lookahead scheduler firing multiple times
    • bpm is wrong until transport control mounts -- now it relies on the $effect of TransportPanel
    • send future clock (futureClock) at onBeat, every
    • onBeat fires twice on audio: true
    • onBeat fires at end
    • figure out next beat + send future clock timea
  • clock.isRunning - is clock running
    • clock.onPlay / onPause / onStop
  • Patchies - 16 Pad Sampler
  • unify audio parameter and float-as-signal visual
  • patch social network: share your own patches (like OpenProcessing)
  • ai.music: accept webcam input
  • Patchies - Object Module Refactor
  • Patchies - Subpatching & Abstraction
  • Patchies - Local Agent CLI
  • cyber subin native node (no iframe)
  • strudel: allow muting (useful when synced to transport)
  • docs: document using msg object as a terminal (shift+enter)
  • Distinguish button and toggle UI
  • multiple metronome sounds for leader and follower
  • beam/sync patches from desktop to mobile for preview
  • Patchies - Spreadsheet Object
  • Patchies - What is Patchies docs
  • Patchies - 15 Minute Screencast
  • Patchies - Tailored Videos for Personas
  • Patchies - Tailored Demos for Personas
  • Patchies - Tailored Guide Docs for Personas
  • Patchies - Add Interactive Help Patches
  • Patchies - Web Audio Modules 2
  • Patchies - Automation Timeline
  • Patchies - Preset Library Repositories
  • Patchies - Projection Mapping
  • Patchies - Edit and Perform Modes
  • Patchies - Figure out versioning
  • Patchies - Update Special Thanks
  • Patchies - Headless Patcher API
  • Patchies - Code Editor Layouts
  • bug: slider value cannot be changed or dragged on patch lock
  • drag: drag into empty space to insert (like n8n)
  • edge: click on center of edge to add an in-between node
  • help: open doc links in the in-app doc browser + navigation
  • fix: issues with service workers and offline mode
  • Changelog & Newsletter
  • sfz sample support?
    • recommended by kijjaz
  • giscus comment section in documentation pages
  • bug - Control only shows trigger on Safari iPhone
  • Patchies - MIDI Clock Sync
  • db object for document store
  • db instance for document store in JS
  • real-time live collaboration
  • Patchies - Option+Enter Quick Insert Action
  • Patchies - Remote Executors
  • Patchies - WebGPU rendering
  • Patchies - Embedded Patchies
  • Patchies - noInputImage for GLSL and Hydra Operators
  • add pub’s logo into patchies web and documentation
  • picture-in-picture (PIP) API -- turn objects into PIP widgets e.g. canvas.dom
  • numeric slider in code editor
    • glsl
    • js
  • allow registering custom svelte-flow visual nodes in embedding mode
  • bug: we did not destroy and cleanup properly when switching between demo patches -- weird inlets in strudel, audio not playing
  • share it with the world!
    • reddit forums
    • share bad apple with asm block
    • create discord channel
    • shader graph demo
    • hacker news
  • add patcher / global settings page
  • snap to grid should be configurable via settings
  • knowledge graph
  • codemirror completion: GLSL
  • codemirror completion: Hydra
  • codemirror completion: Strudel
  • codemirror completion: Assembly
  • csound~: better integration
    • proper message handling
    • supporting audio inputs
    • dynamic inlets/outlets
  • usb: native webusb object
  • background templates
    • selectable templates from PatternCraft.fun
    • better default color schemes - dark and light mode (like Tweakable)
  • dynamic theming system
    • make themes globally styleable
    • use CSS variables, class names and tailwind
    • e.g. set background and text colors
  • visual overhauls?
  • grouping block - for providing sections
    • recommended by Chun
  • color picker in code editor
    • glsl
    • js
  • x/y pad in code editor
    • glsl
    • js
  • auto-refresh P5.js
    • like in p5 website editor, no need to hit run
  • object: show sliders or numeric input on text object parameter click
    • Drag up and down to change number (like PureData)
    • If min/max defined -> show slider range
    • If no min/max -> show just text input
  • message: allow dragging up/down if message is a pure number, like in puredata
    • no need for sliders
  • easier midi parameter mapping?
  • create user-made text objects using the object service
  • create user-made custom visual objects with svelte or js.
  • LLM Video Generation e.g. stream diffusion
  • dynamically sized video objects - right now you can only have one size
  • rust webassembly plugin system
  • add value inlets $1 to $9 to some blocks?
    • easier than using recv for simple value cases
  • look at awesome-creative-coding and see cool stuff we can add
  • virtual players (like renick bell?)
Back to Home