esp3d_core

The esp3d_core module is the foundational layer of the ESP3D-X firmware. It owns the firmware’s bootstrap lifecycle, all persistent settings, the inter-module message bus, the [ESP...] command dispatcher, the hardware abstraction layer, LVGL display utilities (including screen snapshot), a system-wide diagnostic message history, and a suite of memory-safe string helpers. Every other module depends on services defined here.

Architecture Overview
Components
Component Dependency Graph
Initialization Sequence
Message Bus Data Flow
Settings System Flow
Snapshot Capture Flow
Memory & Thread-Safety Constraints
Key Types Reference
Related Documentation

Architecture Overview

Components

ESP3DX — Bootstrap

File: main/core/includes/esp3d_x.h

ESP3DX is the single top-level lifecycle owner of the entire firmware. It is instantiated once (main/main.cpp → app_main) and exposes only two methods:

Method	Description
`begin()`	Initialises all subsystems in dependency order (see below)
`end()`	Stub, returns `true` (nothing is shut down)

The class is marked final — it is not meant to be subclassed. It acts as a composition root: nothing about the internal startup order leaks to other modules.

Notes on the current order:

The reset reason is logged at error level on purpose (visible at any log level).
A pending update (firmware, config, resources, resource patches) is only detected before the UI starts; with a display, the update screen runs it after the first frame is rendered (waitFirstFrameRendered()); without a display, it runs log-only. The Small UI tier has no update screen: it shows the progress through the update_progress / update_status values.
USB CDC device (ESP3D_USB_CDC_DEVICE_FEATURE): when it is not the CNC output, the PC command client on CDC #0 starts after the stream; a failure does not fail the boot. [ESP935]DISABLE keeps the client stopped but the driver stays installed (the log port on CDC #1 keeps working). When it is the CNC output ([ESP950]USB-CDC), the stream already started it.
esp3d_heap_milestone() records the heap after each stage (after_nvs_board, after_fs, after_lvgl_ui, after_gcode_stream, before_network_task, boot_complete).

See entry_point.md for app_main context.

ESP3DSettings — NVS Persistent Settings

Files: main/core/includes/esp3d_settings.h

ESP3DSettings stores all firmware configuration in ESP-IDF’s Non-Volatile Storage (NVS). It is the single source of truth for all persistent state — no other module writes to NVS directly.

Design principles

Type-safe access — separate read/write methods per type (byte, uint32, string, IP).
Thread-safe writes — a pthread_mutex_t serialises NVS write+commit sequences and reset(). Reads are lock-free because NVS itself is internally thread-safe and returns either the committed old or new value, never a torn read.
Schema versioning — checkSchemaVersion() detects stale NVS layouts after a firmware upgrade and triggers a reset to defaults. reset() itself writes the schema version once the defaults are stored: without it, the next boot saw version 0 (major mismatch) and reset again, losing every setting changed since the first reset (e.g. the output client chosen on the first boot after a flash erase).
Validation — every public write is gated by a validation method before touching NVS. Examples of build-aware rules:
- esp3d_radio_mode only accepts the radio modes compiled in the build (off, WiFi STA / AP / setup AP, BT serial, BLE); esp3d_fallback_mode (WiFi builds) only accepts off, STA, setup AP, limited AP and BT serial;
- jog step / feedrate lists (esp3d_<axis>_jog_steps / _jog_feedrates) accept the mm / in section tags (case-insensitive) besides numbers, see jog_unit_lists.md;
- esp3d_jog_wheel_sensitivity (encoder builds, CNC targets) accepts 0–100 (0 = classic step jog).
IP settings — readUint32() also reads ip_t settings in WiFi builds (and returns the default IP on failure); the settings API used to return a wrong value for them.

Setting index

Settings are enumerated in ESP3DSettingIndex (generated from .inc X-macro files). This keeps the header short while allowing board-specific target settings to extend the enum without modifying shared code.

Setting types

`ESP3DSettingType`	Storage	Notes
`byte_t`	1 byte	Enums, flags
`integer_t`	4 bytes	Port numbers, baud rates
`string_t`	variable	SSIDs, hostnames, passwords
`ip_t`	4 bytes	IP stored as `uint32_t`
`float_t`	≤15 chars	Stored as ASCII string
`mask` / `bitsfield`	variable	Feature flags

Global instance

extern ESP3DSettings esp3dXsettings;

ESP3DJsonSettings — INI / JSON Config Parser

File: main/core/includes/esp3d_json_settings.h

ESP3DJsonSettings reads and writes section/key values from INI-style configuration files (stored on flash or SD). It is used by the update service to apply settings from a user-supplied config file without a full flash.

Method	Description
`readString(section, key)`	Returns the string value for `[section] key =`
`writeString(section, key, value)`	Persists a value to the file

A pthread_mutex_t protects the internal parse state (_value, _valueIndex) across the full duration of both readString and writeString — the mutex must be held while iterating the file because two tasks could otherwise corrupt the shared parser state.

ESP3DParseError communicates structured failure reasons back to callers instead of a plain boolean.

extern ESP3DJsonSettings esp3dXJsonSettings;

Related: Storage & Configuration module (update service uses this to apply INI patches from SD).

ESP3DClient & ESP3DMessage — Message Bus

File: main/core/includes/esp3d_client.h

These two types are the backbone of all inter-module communication in the firmware. Every transport (Serial, BT, USB, Socket, WebSocket) and every service that produces or consumes data inherits from ESP3DClient.

ESP3DMessage

A single data-carrying packet passed between components:

struct ESP3DMessage {
    uint8_t*                 data;
    size_t                   size;
    ESP3DClientType          origin;       // Sender identity
    ESP3DClientType          target;       // Intended receiver
    ESP3DAuthenticationLevel authentication_level;
    ESP3DRequest             request_id;   // HTTP request handle or numeric ID
    ESP3DMessageType         type;         // head | core | tail | unique
    ESP3DMessagePriority     priority;     // normal → back of queue; high → front
};

origin / target use ESP3DClientType (an enum of all transport/service types).
type supports multi-part streaming: head opens a response, core carries payload chunks, tail closes it, unique is a self-contained message.
Priority: high messages are pushed to the front of the TX queue via addFrontTxData(), giving them immediate dispatch ahead of queued normal messages.

ESP3DClient

A bidirectional queue manager. Each concrete transport subclasses it:

See Communication_Transports.md for transport implementations.

Queue discipline

Max queue sizes are set per-client via setRxMaxSize() / setTxMaxSize() (1024 bytes by default).
purgeRxByOrigin() / purgeTxByOrigin() allow selective eviction when a transport disconnects.
addRxData() / addTxData() catch std::bad_alloc from the deque and log total free + largest block.

Low-memory admission guard

Every transport calls ESP3DClient::rxAdmit(origin, data, size) in its pushMsgToRxQueue(), before any allocation. Thresholds (main/core/esp3d_client.cpp): reserve 6144 B free / 2048 B largest block; hard floor for the CNC link 3072 / 1024 B.

Origin	Heap above reserve	Below reserve	Below the CNC hard floor
CNC link (output client)	admitted	only vital lines (`ok`, `error`, `ALARM:`, `Grbl…` banner, `<…>` status reports) are admitted; echoes, `[MSG:…]`, `[GC:…]` are skipped	vital lines refused too: memory saturation (overflow)
Any other input	admitted	refused; inside a burst (another line < 500 ms before) or the 3rd refusal within 1 s → overflow, otherwise only this line is dropped (isolated drop)	—

An overflow (esp3d_rx_overflow_raise()) closes the input: every input except the CNC link is refused until Restart / Recover (esp3d_rx_input_reopen()), single safety realtime commands (! ~ ? 0x18 0x85, esp3d_rx_is_safety_realtime()) excepted. The same file holds the other allocation-free lost-line flags read by the gcode host task: esp3d_rx_line_lost() / esp3d_rx_buffer_lost() (line lost after admission, per origin), esp3d_tx_line_lost() (line for the controller not sent). Host reactions: CNC_Firmware_Integration.md “Lost Lines, Input Overflow and Resets”; design: esp32_memory_constraints.md §3.

ESP3DCommands — Command Dispatcher

File: main/core/includes/esp3d_commands.h

ESP3DCommands parses and routes [ESPNNN] commands received from any transport. It is the unified entry point for all firmware configuration and control via the ESP3D command set.

Dispatch flow

is_esp_command() scans the message buffer for the [ESP prefix.
get_param() / hasTag() extract named parameters from the command string.
dispatch() is overloaded to accept raw buffers, C strings, or a full ESP3DMessage*.
dispatchAnswer() formats JSON or plain-text response messages.
_lastESP3DCmdOrigin swallows the trailing \n/\r only on the same transport that sent the command — preventing a stray newline on Serial from being silently dropped because of a BT command.
checkCommandChannelLine(level, msg, size) (authentication builds) gates the command channels that have no session record — the UART0 console and the USB CDC device client keep their own level. A pwd= on an ESP command sets the level (a wrong password gives guest); on another line it can only log a guest in. The line is refused when the level is guest and it is not an ESP command. See esp3d_x_stream_authentication.md §2.2.
Serial debug echo of the CNC lines (when serial is not the output client): status reports <…> from the socket / WebSocket client are not echoed (they arrive every ~200 ms), except the answer to a ? typed on the serial console within the last second; a bare line ending is skipped only when it follows a skipped line (FluidNC sends \r\n, split in two by the transport).
ESP3DClientType::usb_cdc_device (18) is the native USB CDC device client (ESP3D_USB_CDC_DEVICE_FEATURE): dispatch() routes to usbCdcDeviceClient.process() and refuses the message when the client is not started.

Command range overview

Range	Area
ESP0	Help / command list
ESP100–ESP115	Radio and network setup (STA / AP, IP, hostname, radio mode, `[ESP109]` STA rejoin from the setup fallback)
ESP120–ESP135	HTTP, socket server, CNC-over-IP client (TCP socket / WebSocket)
ESP140	Time / NTP
ESP160–ESP190	WebSocket server, camera, WebDAV
ESP200–ESP216	SD card, sensor, screen status output, display snapshot
ESP250–ESP290	Sound, brightness, orientation, theme, delay
ESP300–ESP301	Lua scripts (run, status / control)
ESP400–ESP460	Settings, AP scan, status (`[ESP420]`, incl. build variant), LVGL memory pool, `[ESP444]` RESET / RESTART / FACTORY (boot to the factory app), ESP3D and CNC-over-IP discovery (mDNS), UI language
ESP500–ESP555	Authentication / passwords
ESP600–ESP610	Push notifications
ESP700–ESP800	Flash / SD / global file systems, file streaming and its scripts, firmware version
ESP900–ESP950	Serial and USB serial, buzzer, native USB CDC device client (`[ESP935]`), default output client (`[ESP950]`, incl. `USB-OTG` / `USB-CDC`)

Individual command implementations live in esp3d_commands.md.

Global instance

extern ESP3DCommands esp3dCommands;

esp3d_hal / Esp3dTimout — Hardware Abstraction

File: main/core/includes/esp3d_hal.h

The esp3d_hal namespace wraps FreeRTOS timing primitives so the rest of the firmware does not directly call ESP-IDF tick APIs.

Function / Class	Description
`esp3d_hal::millis()`	Milliseconds since boot (`int64_t`)
`esp3d_hal::micros()`	Microseconds since boot (`int64_t`)
`esp3d_hal::seconds()`	Seconds since boot (`int64_t`)
`esp3d_hal::getEfuseMac()`	Unique 64-bit chip MAC address
`esp3d_hal::wait(ms)`	Blocking delay — minimum `ESP3D_MINIMAL_WAIT` (10 ms = 1 FreeRTOS tick at 100 Hz). `wait(0)` = pure `taskYIELD()`. A value below one tick is only a yield and logs an error (tripwire)
`esp3d_hal::resetReason()`	Readable reason of the last reset (power-on, external pin, software restart, PANIC, interrupt / task / other watchdog, deep sleep, BROWNOUT, SDIO). Logged at boot and reported by `[ESP420]` debug, to tell a cable / supply problem from a crash or a watchdog
`Esp3dTimout`	RAII timeout object

`Esp3dTimout`

class Esp3dTimout final {
public:
    Esp3dTimout(int64_t timeout);   // timeout in milliseconds
    bool isTimeout();               // true if elapsed > timeout
    void reset();                   // restart the timer
};

Typical usage:

Esp3dTimout t(5000); // 5-second timeout
while (!t.isTimeout()) {
    if (done()) { break; }
    esp3d_hal::wait(ESP3D_MINIMAL_WAIT);
}

Wait constants

Constant	Value	Meaning
`ESP3D_MINIMAL_WAIT`	10 ms	Shortest real sleep (1 FreeRTOS tick)
`ESP3D_ONLY_YIELD`	0 ms	Pure yield — no delay, no IDLE feed

LVGL Utilities — Snapshot & Timer Control

Files: main/core/esp3d_lvgl.cpp, main/core/includes/esp3d_lvgl.h

These utilities bridge the LVGL display engine with the rest of the firmware for two specific needs: pausing all LVGL timers before a screenshot, and capturing a full-screen snapshot to an SD file.

⚠️ All LVGL operations run on Core 1 under the LVGL task. These utilities must never be called from a high-frequency ISR or a non-LVGL task without proper synchronisation.

Timer management

Function	Description
`lv_timer_pause_all()`	Iterates all active LVGL timers and pauses each; saves their handles (up to `MAX_TRACKED_TIMERS = 32`). Idempotent — safe to call when already paused.
`lv_timer_resume_all()`	Resumes only timers paused by the previous call; validates each handle with `lv_timer_is_valid()` before resuming.
`lv_timer_is_valid(timer)`	Scans the LVGL timer list to verify a handle is still alive.

Snapshot feature (`ESP3D_SNAPSHOT_FEATURE`)

A mutex-protected state machine captures every LVGL flush callback pixel into a binary file on SD:

File format (raw_header_t):

typedef struct {
    char     signature[4];  // "E3D\0"
    uint32_t width;         // horizontal resolution
    uint32_t height;        // vertical resolution
    uint32_t format;        // 0 = RGB565, 1 = RGB888
    uint32_t reserved;
} __attribute__((packed)) raw_header_t;

Pixel data follows the header as raw packed pixels (2 bytes/pixel for RGB565, 4 bytes/pixel for RGB888). The converter tools in tools/images_converter/ can turn this into PNG.

Known LVGL quirk: with the ILI9341 driver at LV_COLOR_DEPTH == 16, sizeof(lv_color_t) reports 3 bytes (wrong). The snapshot code forces bytes_per_pixel = 2 for RGB565. Re-check on any LVGL upgrade.

API	Description
`esp3d_snapshot_init()`	One-time mutex creation
`esp3d_snapshot(filename)`	Trigger capture; blocks until done or timeout
`esp3d_snapshot_is_active()`	Poll from other tasks
`esp3d_snapshot_abort()`	Cancel an in-progress capture
`esp3d_snapshot_deinit()`	Free mutex
`esp3d_snapshot_estimate_size()`	Compute expected file size before capture

System Message History

Files: main/core/esp3d_system_message_history.cpp, main/core/includes/esp3d_system_message_history.h

A thread-safe ring buffer (max 20 entries) of pendant-local diagnostic messages — distinct from CNC firmware messages. Used by the UI to surface boot warnings, resource-load failures, and theme-customisation errors.

Message types

enum class ESP3DSystemMessageType : uint8_t {
    info    = 0,
    warning = 1,
    error   = 2
};

struct ESP3DSystemMessage {
    ESP3DSystemMessageType type;
    std::string content;
};

API

Function	Description
`esp3d_system_message_add(type, content)`	Append to ring buffer; notifies UI via `ESP3DValues`
`esp3d_system_message_clear()`	Empty buffer; clears UI value
`esp3d_system_message_get_history()`	Returns a snapshot copy under lock — never iterate the live deque directly
`esp3d_system_message_hook_log_errors()`	Register an `esp3d_log` hook: every `esp3d_log_e()` call anywhere in firmware is automatically forwarded here as an `error` message

Allocation failures are absorbed: std::bad_alloc on push_back or on the snapshot copy is caught and only counted (s_drop_count). The add path does not log anything itself — it runs from the log error hook while the log mutex is held, so calling esp3d_log there would self-deadlock.

Integration with ESP3DValues

Every add / clear operation calls:

esp3dXValues.set_value(ESP3DValuesIndex::local_message_history, content);

This triggers any UI subscriber (e.g. the firmware status screen) without polling. The mutex protecting the ring buffer is never held during the set_value() call — the Values system has its own lock, and nested locking would deadlock.

See esp3d_log.md for the hook registration API. See values.md for the observable system.

esp3d_string — String Utilities

File: main/core/esp3d_string.cpp

A collection of helper functions used throughout the firmware for string formatting and manipulation. All functions that return pointers use thread_local static buffers — returned pointers remain valid until the same function is called again from the same thread. Never store these pointers across tasks or across a second call.

Functions

Function	Description
`set_precision(str, precision)`	Format a float string to `N` decimal places
`str_replace(str, old, new)`	Replace all occurrences of a substring
`str_trim(str)`	Strip leading and trailing whitespace
`str_toUpperCase(str*)`	In-place uppercase (`std::string`)
`str_toLowerCase(str*)`	In-place lowercase (`std::string`)
`formatBytes(bytes)`	Human-readable size string (`B`, `KB`, `MB`, `GB`)
`urlDecode(text)`	Percent-decode a URL string
`urlEncode(text)`	Percent-encode a URL string
`endsWith(str, endpart)`	Suffix check
`startsWith(str, startPart)`	Prefix check
`getContentType(filename)`	Map file extension to MIME type
`find(str, sub, start)`	Forward substring search
`rfind(str, sub, start)`	Reverse substring search
`getPathFromString(str)`	Extract directory path (strips filename)
`getFilenameFromString(str)`	Extract filename (handles trailing `/`)
`getTimeString(time, isGMT)`	Format `time_t` as RFC 1123 (GMT) or ISO 8601 (local)
`generateUUID(seed)`	Generate a pseudo-random 36-char UUID
`expandString(s, formatspace)`	Expand `%ESP_IP%`, `%ESP_NAME%`, `%ESP_DATETIME%` macros
`char_toUpperCase(char*)`	In-place uppercase (C string)

Thread-safety model

Each FreeRTOS task has its own copy of every thread_local buffer. Two tasks can call the same helper concurrently without races. Within one task, a second call overwrites the previous result.

⚠️ std::string operations in these utilities can throw std::bad_alloc on heap exhaustion. All functions catch this and return a safe fallback (usually the original input), in compliance with the project’s Memory Constraints.

Component Dependency Graph

Initialization Sequence

ESP3DCommands is a global instance with no explicit start step. The full order, with the optional services, is in ESP3DX — Bootstrap.

Message Bus Data Flow

Settings System Flow

Snapshot Capture Flow

Memory & Thread-Safety Constraints

Concern	Implementation
NVS write atomicity	`pthread_mutex_t` in `ESP3DSettings` — non-recursive, held only during `set + commit` or `reset()`
Message queue access	Per-client `pthread_mutex_t*` for both RX and TX queues
System message history	`pthread_mutex_t` — never held while calling `esp3dXValues.set_value()`
JSON settings parse state	`pthread_mutex_t` for full duration of `readString` / `writeString`
Snapshot state	`SemaphoreHandle_t` FreeRTOS mutex — 1-second timeout on acquire
String utilities	`thread_local` static buffers — one copy per FreeRTOS task, no sharing
`std::string` in hot paths	Avoided or wrapped in `try/catch(std::bad_alloc)` with safe fallback
LVGL operations	Must run on Core 1 LVGL task only; never call from Core 0 or ISR
Dynamic allocation	Heap checks required — worst case ~10 KB available in BT mode

See esp32_memory_constraints.md for the full heap fragmentation playbook.

Key Types Reference

Type	Header	Purpose
`ESP3DX`	`esp3d_x.h`	Firmware lifecycle root
`ESP3DSettings`	`esp3d_settings.h`	NVS settings access
`ESP3DSettingIndex`	`esp3d_settings.h`	Typed setting identifiers
`ESP3DSettingDescription`	`esp3d_settings.h`	Per-setting metadata
`ESP3DJsonSettings`	`esp3d_json_settings.h`	INI/JSON config parser
`ESP3DClient`	`esp3d_client.h`	Message queue base class
`ESP3DMessage`	`esp3d_client.h`	Message carrier
`ESP3DMessageType`	`esp3d_client.h`	`head/core/tail/unique`
`ESP3DMessagePriority`	(client types)	`normal/high`
`ESP3DCommands`	`esp3d_commands.h`	`[ESP...]` command router
`Esp3dTimout`	`esp3d_hal.h`	RAII timeout utility
`raw_header_t`	`esp3d_lvgl.h`	Snapshot file header
`ESP3DSystemMessage`	`esp3d_system_message_history.h`	Diagnostic message record
`ESP3DSystemMessageType`	`esp3d_system_message_history.h`	`info/warning/error`
`ESP3DParseError`	`esp3d_json_settings.h`	JSON parse error codes

Document	Relationship
entry_point.md	Calls `ESP3DX::begin()` — firmware startup
esp3d_commands.md	Implements `ESP0`–`ESP950` command handlers
esp3d_log.md	Logging infrastructure; `esp3d_core` registers an error hook
values.md	`ESP3DValues` observable system — receives system message notifications
esp3d_activity_manager.md	Activity tracking, uses `ESP3DClient` message flow
translations.md	Translation service, called by UI using `esp3d_string` helpers
Communication_Transports.md	All transports subclass `ESP3DClient`
ui_core.md	LVGL UI uses snapshot and timer utilities from this module
esp32_memory_constraints.md	Heap / fragmentation constraints that govern allocation rules here
esp3d_log_guide.md	`esp3d_log` macro levels, backends, hook registration

esp3d_core

esp3d_core

Table of Contents

Architecture Overview

Components

ESP3DX — Bootstrap

ESP3DSettings — NVS Persistent Settings

Design principles

Setting index

Setting types

Global instance

ESP3DJsonSettings — INI / JSON Config Parser

ESP3DClient & ESP3DMessage — Message Bus

ESP3DMessage

ESP3DClient

Queue discipline

Low-memory admission guard

ESP3DCommands — Command Dispatcher

Dispatch flow

Command range overview

Global instance

esp3d_hal / Esp3dTimout — Hardware Abstraction

Esp3dTimout

Wait constants

LVGL Utilities — Snapshot & Timer Control

Timer management

Snapshot feature (ESP3D_SNAPSHOT_FEATURE)

System Message History

Message types

API

Integration with ESP3DValues

esp3d_string — String Utilities

Functions

Thread-safety model

Component Dependency Graph

Initialization Sequence

Message Bus Data Flow

Settings System Flow

Snapshot Capture Flow

Memory & Thread-Safety Constraints

Key Types Reference

Related Documentation

`Esp3dTimout`

Snapshot feature (`ESP3D_SNAPSHOT_FEATURE`)