The Problem
ComfyUI users need a ready‑to‑use node set that can turn multilingual lyrics and style tags into full‑length audio tracks. Without a dedicated integration, they must manually stitch together separate language models, tokenizers, and codecs, which is time‑consuming and error‑prone.
What This Does
ComfyUIFL-HeartMuLa supplies a collection of ComfyUI nodes that wrap the HeartMuLa music‑generation models. The core pipeline lives in heartlib/ (e.g., heartlib/pipelines/musicgeneration.py and heartlib/pipelines/lyricstranscription.py) and is exposed through node modules in flnodes/:
flnodes/modelloader.py – downloads, caches, and selects a model variant (3B by default). flnodes/conditioning.py – tokenizes supplied lyrics and style tags via the HeartMuLa tokenizer. flnodes/sampler.py – runs the diffusion‑style sampler with CFG and temperature controls. flnodes/decode.py – converts generated audio tokens to a waveform using the HeartCodec (heartlib/heartcodec/modelingheartcodec.py). flnodes/tagsbuilder.py – builds the tag string expected by the model. flnodes/transcribe.py – runs the reverse pipeline to extract lyrics from existing audio.
Supporting utilities (flutils/audioutils.py, flutils/modelmanager.py) handle file I/O and model lifecycle, while the minimal web UI (web/css/heartmula.css, web/js/heartmulaconditioning.js) demonstrates a front‑end preview.
How To Use It
Setup
Clone into ComfyUI's custom nodes folder cd ComfyUI/customnodes git clone https://github.com/filliptm/ComfyUIFL-HeartMuLa.git cd ComfyUIFL-HeartMuLa
Install Python dependencies
pip install -r requirements.txt
The pyproject.toml lists the same packages, but the repository only provides a requirements.txt; use that for reproducibility.
Configuration
No explicit config files are required. Model files are downloaded automatically by flnodes/modelloader.py to ComfyUI/models/heartmula/. Optional memory‑mode flags (auto, normal, low, ultra) are exposed as a node parameter in the Model Loader UI.
Running
Open ComfyUI and add the FL HeartMuLa Model Loader node; select the 3B model. Connect a FL HeartMuLa Conditioning node and paste lyrics with section markers (e.g., [Verse], [Chorus]). Attach a FL HeartMuLa Tags Builder node to supply genre, vocal, mood, tempo, and instrument tags. Wire the output to FL HeartMuLa Sampler, then to FL HeartMuLa Decode. Connect the final node to a preview/audio output node to listen to the generated track.
All node classes are defined in the flnodes/ package; the entry point for the pipeline is the node graph assembled in the ComfyUI UI, not a standalone script.
Real‑World Use
A podcast production pipeline could automate intro music generation:
from flnodes.modelloader import ModelLoader from flnodes.conditioning import Conditioning from flnodes.sampler import Sampler from flnodes.decode import Decode
loader = ModelLoader(model="3B", memorymode="low") cond = Conditioning(lyrics="[Intro] Welcome to the show", tags="pop, female vocal, upbeat") tokens = Sampler(loader, cond, cfg=2.5, temperature=0.8).run() audio = Decode(loader, tokens).run() audio.save("intro.wav")
The same graph can be built visually in ComfyUI, allowing non‑programmers to tweak tags or structure interactively.
Code Health & Issues
Med – No test suite – repository contains zero tests/ directories or pytest files; functional correctness is unverified. Med – Missing license – no LICENSE file; downstream users lack clear redistribution rights. Low – No lockfile – requirements.txt is present but no requirements-lock.txt or poetry.lock; builds may diverge over time. Low – Limited CI – only a single GitHub Actions workflow (.github/workflows/publish.yml) that publishes the package; no linting, type checking, or test execution. Low – Sparse documentation – README covers usage but there is no API reference for the Python modules; developers must read source to understand function signatures. Low – Hard‑coded paths – model cache path is constructed in flutils/paths.py assuming a standard ComfyUI layout; custom installations may break without modification.
No obvious security risks (e.g., credential files) are present, and the code follows a clear separation between node wrappers and the underlying heartlib pipeline.
The Bottom Line
ComfyUIFL-HeartMuLa delivers a functional set of nodes that let ComfyUI users generate multilingual songs with minimal effort. It is usable out‑of‑the‑box for prototyping or small‑scale production, but the lack of tests, licensing, and reproducible dependency locking makes it a higher‑risk choice for commercial or mission‑critical deployments. Teams comfortable with reviewing and augmenting the code should adopt it; others may prefer a more mature, fully‑tested alternative.