Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop timeline feature enough to complete first video project #255

Open
19 of 30 tasks
Venryx opened this issue Jan 2, 2024 · 0 comments
Open
19 of 30 tasks

Develop timeline feature enough to complete first video project #255

Venryx opened this issue Jan 2, 2024 · 0 comments
Labels
frontend Requiring changes to the frontend code (TypeScript)

Comments

@Venryx
Copy link
Collaborator

Venryx commented Jan 2, 2024

Tasks:

  • MS imported audio-files persist between page refreshes. (while still being stored locally)
  • MS you can assign a point in each audio-file to a timeline-step, as its "start time" in audio track. (stored locally only, but with export of all this info as a JSON file, for eventual re-import/restore functionality to prevent data-loss)
  • MS start-times of steps for audio-file show up on that file's waveform ui.
  • MS step durations can be derived from the start-times assigned in the audio files.
  • MS playback of timeline also plays the associated audio segments.
  • MS the timeline/audio panel can import multiple audio files at once, and have a button to auto-assign them to the timeline steps based on matches between the step-ids and the filenames.
  • MS step->audio associations let you assign a volume-multiplier as well.
  • Add a visual "talking-time tracker" feature, which is visible to editors, and in the video export (if/when enabled). [note that some segments should be excluded, eg. intro at start -- perhaps marked using the full/left/right positioning flags for now]
  • Fix the issue where focus-nodes can apparently be "active" (and thus affect auto-scrolling), even when the given nodes are not actually visible in the main/non-load-helper map. [optional extra task: make-so a warning is given when the steps' fields try to do this]
  • Rework the timeline Editor and Playing panels to have the switching be less jarring. (probably get rid of the "tab" approach, and instead have more granular toggles for specific functionality)
  • Replaced node-reveals system with a more expansive step-effects system.
  • MS the "time_relative" field of step-effects is processed/not-ignored. (probably requires a moderate rework, to have the accessors all based on time or reached-effect-index, rather than reached-step-index as it is now)
  • Improve performance of processing that occurs while timeline is progressing. (for some cases at least, it's still not quite as fast as it was before the rework -- but the cleanliness [+reliability] improvements outweigh the likely slight-perf-loss)
  • Start work on Electron wrapper app, for making interaction with the local filesystem more streamlined.
  • MS electron app offers a streamlined way to record "takes" of audio (per step), pass them to the voice app, then receive the results + store them + assign them.
  • MS new step-specific audio-file system is fully integrated. (replacing old audio-files-in-map-folder approach)
  • MS electron app has streamlined screenshot-recording functionality. (todo: observability on the saved file/file-ranges)
  • MS electron app has easy way to stitch the recorded screenshot-images and audio files into a combined video.
  • Fix the following frame-recording issue: The frontend occasionally "moves on" and displays the next frame prior to the current frame having been fully captured -- causing "Frame10.png" to show the contents of frame 11, meaning frame 10 and 11 show the same contents, meaning slight jitter in video.
  • Fine-tune the timings of the timeline node-effects, to make sense with the timings of the characters' speech. [project-specific]
  • Add new step-effect type: show on-screen message

Likely tasks:

  • MS TakeUI lets you do basic edits of a recorded audio file. (just trimming for now probably)
  • MS the timeline panel scrolls itself to keep the timeline's current-time arrow in view. (by default anyway)
  • Expand the node-effect struct to allow for (at least) unfocusing of all nodes under a given path.
  • Make a "combined view" in audio-panel, letting you see (and change) time-assignments of each timeline-step, along with the waveforms of all of the audio-files that get triggered throughout the timeline.
  • Add a button to copy the list of ids of each timeline-step (either full-ids or the first X characters of each id).

Possible tasks:

  • Expand electron app to allow easy integration with audio tools. (for more advanced editing)
  • Maybe rework audio-clip system to be just a form of "timeline-step effects".
  • Add way to mass-change the volume-multiplier of step->audio entries based on patterns. (eg. text contains string)
  • Extend timeline-step audio controls row, to allow custom ordering of the audio start-time assignments based on quality. (and perhaps with a star-rating system for knowing when a step has a "good enough" best entry)
@Venryx Venryx added the frontend Requiring changes to the frontend code (TypeScript) label Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
frontend Requiring changes to the frontend code (TypeScript)
Projects
Status: 🔖 Short-term (Venryx)
Development

No branches or pull requests

1 participant