◀ sprint-6
sprint-6

← esp32-spotify-display

Sprint 6 Retro: Album Art Pipeline

Goal: Download album art from Spotify’s CDN, decode the JPEG, apply purple-tinted Bayer dithering, and display it in the now-playing art zone. Every album becomes part of the purple-black visual language. The sad purple rectangle finally earns its keep.

Verdict: The placeholder is dead. Long live the dithered art. Every album cover gets the purple treatment, every track change gets a loading wave, and the 80x80 art zone went from “we’ll get to it” to the prettiest thing on the screen. We even went back and upgraded from 64px thumbnails to 300px source art because life is too short for blocky dithering ╰(°▽°)╯

What We Built

The Full Pipeline

One function call, four stages, two cores:

Core 0                                Core 1
──────                                ──────
spotifyPollTask()                     loop()
  ├─ HTTP poll (existing)               ├─ spotifyPoll() → track data
  ├─ artUrl changed?                    ├─ spotifyConsumeArt()
  │   ├─ httpsGetBinary() → JPEG buf    │   └─ nowPlayingSetAlbumArt(buf)
  │   ├─ jd_prepare + jd_decomp → RGB   │
  │   ├─ ditherAndUpscale() → RGB565    │
  │   └─ signal artNewReady             │
  └─ back to sleep                      └─ LVGL renders from dither buffer

The poll result is written to shared state BEFORE art processing starts. Core 1 gets track info immediately. Art follows 1-2 seconds later. No blocking, no jank.

TJpgDec: The Library That Wasn’t There

LVGL bundles TJpgDec (Tiny JPEG Decompressor, ~1100 lines of C by ChaN). Perfect. Except LVGL guards the entire thing behind #if LV_USE_TJPGD which is 0 in our config. Every type, every function, compiled to nothing.

Solution: yank the source files into lib/tjpgd/, strip the LVGL guards, write a standalone config. Now it’s our library. Three files, zero dependencies, decodes JPEGs into BGR888 pixel blocks via callbacks.

The input callback reads from an in-memory buffer (no filesystem). The output callback copies decoded blocks into a flat pixel array. Both are ~10 lines. TJpgDec is a gem of minimalist C engineering.

Bayer Dithering: The Aesthetic Core

Every pixel goes through the same pipeline:

  1. BGR888 → Grayscale: (77*R + 150*G + 29*B) >> 8 (ITU-R BT.601 luminance)
  2. Bayer threshold: 4x4 ordered dithering matrix, 16 threshold levels (8 to 248)
  3. Binary output: luminance > threshold → purple #9B59B6, else black #000000
  4. RGB565 encode: 16-bit color, native format for both LVGL and the ST7789

The dithering uses source pixel coordinates for the Bayer lookup, not output coordinates. This means the dither pattern scales with the pixel grid, preserving the pattern’s visual frequency regardless of upscale ratio.

Single pass. No temporary buffers beyond the source and destination. 6,400 pixels of pure integer math.

The Resolution Upgrade

First version used Spotify’s 64px thumbnails. It worked. It was… blocky. The dithering was doing its best but you can’t conjure detail from 4,096 pixels.

Spotify serves three sizes: 640px, 300px, 64px. We switched to the 300px image and enabled TJpgDec’s built-in scaling at 1/4 (300 → 75px decoded). Five times the source detail, nearly the same RAM footprint, dramatically richer dithering output.

The scale factor is calculated dynamically: find the largest power-of-two downscale where both dimensions stay >= 64px. Works for any source size Spotify throws at us.

The Loading Wave

When you skip tracks, there’s a 1-2 second gap while Core 0 downloads new art. The old design showed stale art. The new design shows a diagonal sweep of dithered purple dots rippling across the art zone.

Same Bayer matrix, same purple/black palette, same aesthetic. It’s a “loading” animation that looks like it belongs in the dithered world.

int sweep = (int)((millis() / 8) % 200);
int pos = x + y;  // diagonal position
int dist = abs(pos - sweep);
if (dist > 100) dist = 200 - dist;
uint8_t gray = (dist < 35) ? (uint8_t)((35 - dist) * 7) : 0;

Pure integer math. ~0.5ms per frame for 6,400 pixels. Updates the loading buffer in nowPlayingTick(), LVGL renders it via lv_obj_invalidate(). The wave sweeps from corner to corner every ~1.6 seconds with a smooth falloff band.

How the Pieces Connect

Binary HTTP Downloads

The existing httpsGet() returns a String. Strings and JPEG bytes don’t mix. Added httpsGetBinary() to the network module: streams raw bytes via WiFiClient::read() with Content-Length validation, redirect following (CDN URLs bounce), and the same secureClient.stop() + setReuse(false) pattern that keeps our TLS sessions clean.

Shared Art State

Art state piggybacks on the existing sharedResultMutex. Two flags:

  • artNewReady: new dithered buffer available (Core 0 sets, Core 1 clears)
  • artShowPlaceholder: art failed, show purple square (Core 0 sets, Core 1 clears)

spotifyConsumeArt() reads both under mutex, returns the buffer pointer (or nullptr for placeholder). Called every frame from loop(), same pattern as consumePollResult().

Art URL Caching

previousArtUrl lives on Core 0 only. Same URL = no re-download. Different tracks from the same album share an art URL, so switching albums is the only time we fetch. On no-playback, the cached URL clears, so resuming always re-fetches (tiny cost, correct behavior).

Things That Just Worked

  • TJpgDec callbacks. Point the input callback at a memory buffer, point the output callback at a pixel array, call jd_decomp(). The library handles MCU block decomposition, color space conversion, and optional scaling internally. We just shuffle bytes.
  • LVGL lv_image_dsc_t. Fill in the header (magic, color format, dimensions, stride), point .data at our buffer, call lv_image_set_src(). LVGL reads the raw RGB565 pixels directly. No decoding, no caching, no copies.
  • The dithering aesthetic. Bayer ordered dithering with binary purple/black output looks surprisingly gorgeous on a tiny ST7789 screen. Every album cover becomes a purple-tinted pixel art piece. Dark albums get sparse dots. Bright albums get dense fields. The visual language is cohesive and distinctive.
  • Loading wave invalidation. Modifying the pixel buffer in-place and calling lv_obj_invalidate() triggers a re-render from the updated data. No lv_image_set_src() per frame needed. LVGL handles raw formats without caching.

Known Issue: Font Coverage

LVGL’s default Montserrat fonts only include ASCII (U+0020-U+007E) plus FontAwesome symbols. Characters like n with tilde, accented vowels, and other Latin Extended glyphs show as squares. Spanish artists beware. Fix requires generating custom fonts via lv_font_conv with Latin-1 Supplement and Latin Extended-A ranges. Noted in TODO.md for a future sprint.

By the Numbers

MetricValue
Flash usage97.1% (1,272 KB / 1.3 MB)
RAM usage18.0% (59.1 KB / 327 KB)
Flash increase from Sprint 5+0.6% (+7,060 bytes)
RAM increase from Sprint 5+8 bytes (.bss)
Heap budget (art pipeline)~82 KB (JPEG 48K + RGB 19.2K + work 4K + dither 12.8K)
Heap budget (loading anim)12.8 KB
Files created5 (album_art.h/.cpp, lib/tjpgd x3)
Files modified8
New modules2 (album_art, tjpgd)
Source art resolution300x300 (decoded at 1/4 → 75x75)
Dithered output80x80 RGB565
Dither palette2 colors (purple #9B59B6, black #000000)
JPEG download size~15-30 KB per track
Art processing time~1-2s (download + decode + dither)
Loading wave cycle~1.6s per sweep
Compilation errors0 (every build succeeded first try)

What’s Next

Sprint 7: Go-Wild Animations. The static now-playing screen gets personality. Spring physics, particle effects, animated transitions. The screen should feel alive, not just informative.

Sprint 6 complete. The purple rectangle is gone. In its place: a JPEG pipeline that downloads, decodes, dithers, and displays album art in a visual language that makes every cover look like it was born purple. And when it’s loading, it waves. (ノ◕ヮ◕)ノ*:・゚✧