ADR 003: Drop .env Support — Single Configuration Format
Status
Accepted
Update (v10): Musebot v10 renamed the LLM configuration block from
ollama:tollm:and changed host entries from bare strings to objects ({ "url": ..., "token"?, "isLocal"? }), with authentication living only inconfig.jsoncasAuthorization: Bearertokens per host. Key references in this ADR have been updated to thellm.*names; see the v10 LLM Migration Guide for the current configuration surface.
Context
Musebot currently supports two competing configuration formats simultaneously:
- Environment variables (
.envfile withMUSEBOT_*prefix) - loaded byEnvironmentSettings - JSON configuration (
config.jsonorconfig.jsonc) - loaded byConfigLoaderusing thejson5package
This creates complexity, confusion, and maintenance overhead:
- Users must understand and maintain two different format schemas
- Unclear precedence when both files exist
- Validation logic duplicated across both formats
- Risk of accidentally committing secrets with .env files
- Inconsistent configuration validation and error messages
Current environment variable examples:
MUSEBOT_FUNCTION=chat
MUSEBOT_DISCORD_TOKEN=12345
MUSEBOT_OLLAMA_HOSTS=http://host1:11434/,http://host2:11434/
Current config.json examples:
{
"botFunction": "chat",
"discordToken": "12345",
"ollamaHosts": ["http://host1:11434/", "http://host2:11434/"]
}
Decision
Drop .env support entirely and standardize on a unified JSON-based configuration system:
- Remove
.envfile processing — No moreMUSEBOT_*environment variables in code - Unified JSON configuration — Use
config.json(preferred) orconfig.jsonc(with comments support) - Single configuration source — Remove
#read*and#parseDelimitedListmethods fromEnvironmentSettings - Configuration validation at startup — Fail fast with clear error messages for invalid configurations
- Mode property — Rename
botFunctionenum to simplermodeproperty
Rationale
Simplicity: Single configuration format reduces complexity and maintenance burden
Clarity: JSON is easier to understand and edit with modern IDE support (autocomplete, linting, formatting)
Schema validation: Can add JSON schema or runtime validators for complete structure checking
Security: Less risk of accidentally committing secrets with JSON files (though still should never commit actual secrets)
Documentation: Easier to document a single format clearly
Tooling support: Excellent IDE and editor support for JSON configuration; YAML and other formats can be added later if needed
Consequences
Positive
- Reduced confusion: Users only need to learn one configuration format
- Better tooling: IDE autocomplete, validation, and refactoring for JSON
- Simpler code: Remove ~150 lines of environment variable parsing code
- Clearer errors: Single validation path with consistent error messages
- Better security: No risk of .env files being committed to version control
Negative
- Migration burden: Users currently using .env files must manually migrate to JSON
- Breaking change: Existing deployments will break unless they update configuration
- Development friction: Missing some configuration options in JSON during development
- Increased complexity for multi-bot: New
IBotSettingsinterface adds configuration nesting
Neutral
- No performance impact: Configuration is only loaded at startup (once)
- Configuration size: JSON files may be slightly larger but more readable
- IDE requirements: Requires a code editor with JSON support (standard for modern development)
Implementation Plan
Phase 1: New Configuration Schema
Create new interfaces: IBotSettings.ts, IBotMode.ts
BotModeenum (new file)export enum BotMode { Chat = 'chat', Media = 'media' }IBotSettingsinterface (new file)export interface IBotSettings { botId: string; nodeEnvironment: string; mode: BotMode; discord: { token: string; channels: string[]; privateMessageUsers: string[]; }; chatApis: { discord: { token: string; channels: string[]; privateMessageUsers: string[]; }; }; comfyUi: { hosts: string[]; }; llm: { hosts: { url: string; token?: string; isLocal?: boolean; }[]; models: string[]; systemPrompt: string; streamsResponse: boolean; }; multiModal?: { randomPrompts: string[]; }; taskQueue?: { numAttempts?: number; retryDelayMs?: number; strategy?: TaskQueueStrategy; forceSerialAcrossHosts?: boolean; }; }New
IBotConfiginterface (update existing file)export interface IBotConfig { bots: IBotSettings[]; }Update
IGlobalSettingsinterfaceexport interface IGlobalSettings { taskQueue: { numAttempts: number; retryDelayMs: number; strategy: TaskQueueStrategy; forceSerialAcrossHosts: boolean; } }
Phase 2: Code Migration
Files to modify:
src/services/environment-settings/IBotConfig.ts- Replace old structure with
bots: IBotSettings[] - Remove old properties that will move to per-bot settings
- Replace old structure with
src/services/environment-settings/IBotSettings.ts(create new)- Define the complete
IBotSettingsinterface
- Define the complete
src/services/environment-settings/IBotMode.ts(create new)- Export
BotModeenum
- Export
src/services/environment-settings/IGlobalSettings.ts- Add
taskQueueproperty to structure
- Add
src/services/environment-settings/IEnvironmentSettings.ts- Add
botMode: BotModeproperties (instead ofbotFunction) - Add method to get active bot settings based on botId
- Add methods to extract settings from per-bot config
- Add
src/services/environment-settings/EnvironmentSettings.ts- Remove sections 145-155:
.envloading logic (if(this.nodeEnvironment !== NodeEnvironment.Test)check) - Remove methods (lines 287-486):
#readEnum<T>()#readDefaultableNumber()#readRequiredString()#readDefaultableString()#parseDelimitedList()#readDelimitedList()#normalizeConfigValue()(deprecated)#readBoolean()#readDefaultableInteger()#readDefaultableRangedInteger()#mapLegacyFunctionsToCurrent()
- Replace with direct mapping from config object to properties
- Add validation for
MUSEBOT_*environment variables (throw if detected) - Add priority logic: if both
config.jsonandconfig.jsoncexist, log warning and useconfig.jsonc
- Remove sections 145-155:
src/services/environment-settings/ConfigLoader.ts- Keep config.json vs config.jsonc priority logic (lines 12-29)
- Add warning if both files exist (lines 20-23): log at debug level
src/services/ServiceContainer.ts- Update to create
IBotSettingsfrom loaded config - Add method to get bot ID
- Update method to get active bot settings
- Update to create
src/app.ts- Update config loading flow
- Pass bot settings to EnvironmentSettings constructor
- Iterate over bots from config
Phase 3: Validation & Error Handling
Startup validation:
Missing config files:
Error: config.jsonc could not be found or accessed.Both config files exist:
- Warning log:
[ConfigLoader] Warning: Both config.json and config.jsonc exist. Loading config.jsonc (with comments support).Invalid JSON syntax:
Error: config.jsonc exists, but cannot be parsed. Does it contain syntax errors?Configuration validation:
- If mode is 'media' → check
comfyUi.hostshas at least one value, else throw:
Error: Media mode requires at least one ComfyUI host configured in bot.comfyUi.hosts.- If mode is 'chat' → check
llm.hostsandllm.modelshave at least one value, else throw:
Error: Chat mode requires at least one LLM host configured in llm.hosts. Error: Chat mode requires at least one LLM model configured in llm.models.- If
llm.modelsis empty → warning:
Warning: Chat mode has no LLM models configured. Model selection not yet supported.- If mode is 'media' → check
Environment variable detection:
- At startup, iterate through
process.envlooking forMUSEBOT_*keys - If found, throw error with guidance:
[EnvironmentSettings] FATAL ERROR: .env support has been removed. Please migrate all environment variables to config.jsonc format. Migration guide: docs/migration-from-env-to-jsonc.md- At startup, iterate through
Invalid mode:
Error: bot.mode must be one of: 'chat', 'media'Missing required fields:
{ "botId": "required" → throw: botId is required "discord.token" → throw: discord token is required in chat mode "comfyUi.hosts" → throw: ComfyUI hosts are required in media mode "llm.hosts" → throw: LLM hosts are required in chat mode "llm.models" → throw: LLM models are required in chat mode }
Phase 4: Migration Path for Users
Documentation to create:
docs/migration-from-env-to-jsonc.md(new)- Step-by-step migration guide
- Examples showing
.env→config.jsoncmappings - Common configuration scenarios
- Troubleshooting section
config.example.jsonc(new)- Minimal working example with all options documented
- Comments explaining each setting
docs/configuration.md(update existing)- Complete configuration reference
- Explain new structure with diagrams
- Show examples for different bot modes
- Document all configurable options
Example migration:
# .env (old)
MUSEBOT_FUNCTION=chat
MUSEBOT_DISCORD_TOKEN=12345
MUSEBOT_DISCORD_CHANNELS=1343757467854704732
MUSEBOT_OLLAMA_HOSTS=http://host1:11434/,http://host2:11434/
MUSEBOT_OLLAMA_MODELS=model1,model2
MUSEBOT_OLLAMA_SYSTEM_PROMPT=You are a helpful assistant
# config.jsonc (new)
{
"botSettings": {
"botId": "bot-1",
"nodeEnvironment": "development",
"mode": "chat",
"discord": {
"token": "12345"
},
"chatApis": {
"discord": {
"channels": ["1343757467854704732"]
}
},
"llm": {
"hosts": [{ "url": "http://host1:11434/" }, { "url": "http://host2:11434/" }],
"models": ["model1", "model2"],
"systemPrompt": "You are a helpful assistant"
}
}
}
Phase 5: Testing Strategy
Test files to update/create:
src/services/environment-settings/EnvironmentSettings.test.ts- Remove tests for env variable reading
- Add tests for config object mapping
- Add tests for mode validation
- Add tests for missing config files
- Add tests for both config files existing (warning expected)
- Add tests for invalid JSON syntax
src/services/environment-settings/ConfigLoader.test.ts- Update for new config format
- Add tests for both config files priority
- Add tests for config.jsonc priority
src/app.test.ts- Update config loading flow tests
- Add tests for bot settings initialization
src/services/ServiceContainer.test.ts- Update for new bot settings structure
- Add tests for per-bot config extraction
Create new integration tests:
- Test complete configuration flow from file to bot services
- Test validation logic
- Test error messages
Test cases:
- Config only exists at
config.jsonc - Both config files exist (expect warning, use config.jsonc)
- Config only exists at
config.json - Invalid JSON syntax (catch and throw)
- Missing config files (throw with clear message)
- Invalid mode value (throw with list of valid options)
- Mode is 'media', no ComfyUI hosts configured (throw)
- Mode is 'chat', no Ollama hosts configured (throw)
- Mode is 'chat', no Ollama models configured (warning + log)
- Environment variable detected (throw with migration guide)
- Full valid configuration loads successfully
- All bot-specific settings are accessible
Phase 6: Cleanup
Files to remove:
.envfile — Users will delete manually (document in migration guide).env.example— Replace withconfig.example.jsoncsrc/services/environment-settings/constants/EnvironmentKey.ts— No longer needed- Remove
.envexample file reference from documentation
Update documentation:
README.md
- Remove
.envconfiguration instructions - Add JSON configuration instructions
- Add link to migration guide
- Remove
docs/configuration.md (update)
- Remove all
.envexamples - Add JSON examples
- Document new
IBotSettingsstructure
- Remove all
docs/getting-started.md (update)
- Remove
.envsetup instructions - Add
config.jsoncsetup instructions - Include minimal example
- Remove
Any installation/setup guides
- Update to reflect new configuration format
Risks
User migration burden: Users currently using .env must manually migrate
- Mitigation: Comprehensive migration guide with automated helpers, step-by-step instructions, examples for common scenarios, clear error messages during migration
Breaking change: Existing deployments will break
- Mitigation: Clear error messages, extensive documentation, rollback instructions, extensive test coverage
Development friction: Developers missing some config options in JSON
- Mitigation: Use
config.jsoncfor development with comments, comprehensive examples in documentation, IDE autocomplete provides guidance
- Mitigation: Use
Configuration complexity: New multi-bot support adds complexity
- Mitigation: Document bot configuration thoroughly, provide examples, clear error messages for misconfiguration
Tooling dependency: Requires JSON editor support
- Mitigation: JSON is standard format with excellent support in VS Code, JetBrains, and other modern editors
Future Considerations
- JSON Schema Validation: Add schema validation for runtime checks (optional)
- Multiple Configuration Formats: Consider supporting
config.yaml,config.toml, or other formats in the future (optional) - Configuration Validators: Add custom validation service for per-bot specific rules
- Automatic Migration Tool: Create CLI tool to convert .env to config.jsonc automatically (optional)
- Hot Reload: Detect config file changes and reload configuration at runtime (future enhancement)
- Configuration Versioning: Track config format version and validate compatibility (future)
Success Criteria
- ✅
.envfile processing removed from codebase (EnvironmentSettings.ts lines 145-155 removed) - ✅
MUSEBOT_*environment variables trigger error at startup with clear message - ✅ All functionality works with JSON configuration only
- ✅ Both config files logged warning but config.jsonc is loaded (priority)
- ✅ Configuration validation catches missing/invalid settings at startup
- ✅ Users can migrate from .env with clear documentation and examples
- ✅ All tests pass with new configuration system (expecting ~269 tests)
- ✅ No breaking changes to internal bot functionality
- ✅ Migration guide completed and published
- ✅
config.example.jsonccreated with complete examples
Related
- ADR 001: Multi-Instance Support via Container Hierarchy
- ADR 002: Config Format Compatibility (complementary, not dependent)
- Configuration documentation (
docs/configuration.md) - Migration guide (
docs/migration-from-env-to-jsonc.md) config.jsonschemaIBotConfiginterfaceEnvironmentSettingsimplementationConfigLoaderutility