Configuration system #49

Merged

cswimr merged 56 commits from feat/config/init into main 2025-12-08 17:15:23 -05:00

Conversation 1 Commits 56 Files changed 15 +1562 -109

cswimr commented 2025-11-10 15:29:51 -05:00

Owner

Copy link

I would like to write a thin, type-safe (to the best of my ability) wrapper around Red-DiscordBot's Config framework.

Red's Config framework is awesome, and works well enough as is. It's really easy to use and understand, and that's a good thing. You don't have the overhead of learning something like Piccolo or (shiver) SQLAlchemy just to store some data, just toss it into Config!

However, Config does have quite a few quirks that make it less easy to use, often in ways that are not immediately obvious to newer cog developers. Here are some of what I would consider pressing issues with Red's existing Config framework.

• No type safety or data validation. You have no idea what is actually being returned from Config unless you manually check its type. Blowing up on unexpected data is fine, but this should be controlled. Letting unexpected data get down the line into your actual code is dangerous, and can cause issues that are nigh impossible to reproduce or debug. Instead, the configuration framework should blow up on a validation step outside of the actual cog's logic, so that there's no risk of unintended behavior due to an improper type being passed.
  • Validate the type of configuration values using Pydantic, at both set-time and get-time. Also allows for constraints where possible, so regex matching for strings, gt/lt for integers, etc.
  • Allow using Pydantic models as a type, so nested data structures can exist.
• No static analysis support. Config uses attributes that don't actually exist on the Config class prior to runtime (as far as I understand, not super familiar with the internals). This means that your IDE / type-checker / linting tool has no way of knowing if an attribute you're providing actually exists or not. force_registration=True is technically a sort-of workaround to issues this causes at runtime, but this does nothing for tooling that utilizes static analysis.
  • Ensure everything a developer expects to exist does so statically, so as to allow for static analysis and IDE integration.
• Unscalable configuration UX for end-users, or what I like to call "configuration command sprawl". I've noticed this a lot in my own cogs, and even done some (albeit ugly) work to "solve" it. This is when a cog has too many configuration commands, and makes it difficult for a new user to get up to speed with the cog's featureset.
  • Offer a configuration menu, likely utilizing Discord's Components V2 system.

Just as an example, this is what a cog utilizing this module might look like today. I would read this as if it were psuedo-code. Note that this is liable to change over time, and none of this module's structure is finalized. (In fact, as of writing, it doesn't actually work yet... pydantic issues... sigh)

from typing import Annotated, ClassVar, reveal_type

from pydantic import PositiveInt
from redbot.core import commands
from tidegear import BaseConfigSchema, Cog, ConfigMeta, GlobalConfigOption, GuildConfigOption
from typing_extensions import override


class ExampleConfigSchema(BaseConfigSchema):
    version: ClassVar[PositiveInt] = 1

    example_global_option: Annotated[GlobalConfigOption[str], ConfigMeta(default="example string")]
    example_guild_option: Annotated[GuildConfigOption[int], ConfigMeta(default=100)]


class ExampleCog(Cog):
    config: ExampleConfigSchema
    _config_identifier: int = 1234567890  # Consider using your Discord ID for this, instead of a random assortment of numbers.

    @override
    async def cog_load(self) -> None:
        await super().cog_load()  # This is important! Tidegear has its own logic within the `cog_load` method that expects to be run.
        self.config = await ExampleConfigSchema.init(cog_name=self.metadata.name, identifier=self._config_identifier, logger=self.logger)

    @commands.is_owner()
    @commands.command()
    async def global_config(self, ctx: commands.Context, *, value: str | None = None) -> None:
        if value:
            # Works
            await self.config.example_global_option.set(value)
            # ValidationError: Incorrect type, expected `int` (got `list[int]`)
            await self.config.example_global_option.set([value])

            await ctx.send(f"Set `{self.config.example_global_option.key}` to `{value}`")
        else:
            current_value = await self.config.example_global_option.get()
            # OR
            current_value = await self.config.example_global_option()
            # Both of these lines work and do the same thing.
            # The `__call__` interface is provided to be more familiar to Red Config users.

            reveal_type(current_value)  # str
            await ctx.send(f"The current value of the `{self.config.example_global_option.key}` setting is `{current_value}`.")

    @commands.admin()
    @commands.guild_only()
    @commands.command()
    async def guild_config(self, ctx: commands.GuildContext, *, value: int | None = None) -> None:
        if value:
            await self.config.example_guild_option.set(ctx.guild, value)
            await ctx.send(f"Set `{self.config.example_guild_option.key}` to `{value}`")
        else:
            current_value = await self.config.example_guild_option.get(ctx.guild)
            reveal_type(current_value)  # int
            await ctx.send(f"The current value of the `{self.config.example_guild_option.key}` setting is `{current_value}`.")

I would like to write a thin, type-safe (to the best of my ability) wrapper around Red-DiscordBot's [`Config`](https://docs.discord.red/en/stable/framework_config.html) framework. Red's Config framework is awesome, and works well enough as is. It's really easy to use and understand, and that's a good thing. You don't have the overhead of learning something like Piccolo or *(shiver)* SQLAlchemy just to store some data, just toss it into Config! However, Config does have quite a few quirks that make it less easy to use, often in ways that are not immediately obvious to newer cog developers. Here are some of what I would consider pressing issues with Red's existing Config framework. - **No type safety or data validation.** You have no idea what is *actually* being returned from Config unless you manually check its type. Blowing up on unexpected data is fine, but this should be controlled. Letting unexpected data get down the line into your actual code is dangerous, and can cause issues that are nigh impossible to reproduce or debug. Instead, the configuration framework should blow up on a validation step outside of the actual cog's logic, so that there's no risk of unintended behavior due to an improper type being passed. - [x] Validate the type of configuration values using [Pydantic](https://pydantic.dev/), at both set-time and get-time. Also allows for constraints where possible, so regex matching for strings, gt/lt for integers, etc. - [x] Allow using [Pydantic models](https://tidegear.csw.im/stable/ref/pydantic/#tidegear.pydantic.BaseModel) as a type, so nested data structures can exist. - **No static analysis support.** Config uses attributes that don't actually exist on the Config class prior to runtime (as far as I understand, not super familiar with the internals). This means that your IDE / type-checker / linting tool has no way of knowing if an attribute you're providing actually exists or not. [`force_registration=True`](https://docs.discord.red/en/stable/framework_config.html#tutorial) is technically a sort-of workaround to issues this causes at runtime, but this does nothing for tooling that utilizes static analysis. - [x] Ensure everything a developer expects to exist does so statically, so as to allow for static analysis and IDE integration. - **Unscalable configuration UX for end-users**, or what I like to call *"configuration command sprawl"*. I've noticed this a lot in my own cogs, and even done some (albeit ugly) work to "solve" it. This is when a cog has too many configuration commands, and makes it difficult for a new user to get up to speed with the cog's featureset. - [ ] Offer a configuration menu, likely utilizing Discord's Components V2 system. Just as an example, this is what a cog utilizing this module might look like today. **I would read this as if it were psuedo-code.** Note that this is liable to change over time, and none of this module's structure is finalized. (In fact, as of writing, it doesn't actually *work* yet... pydantic issues... *sigh*) ```py from typing import Annotated, ClassVar, reveal_type from pydantic import PositiveInt from redbot.core import commands from tidegear import BaseConfigSchema, Cog, ConfigMeta, GlobalConfigOption, GuildConfigOption from typing_extensions import override class ExampleConfigSchema(BaseConfigSchema): version: ClassVar[PositiveInt] = 1 example_global_option: Annotated[GlobalConfigOption[str], ConfigMeta(default="example string")] example_guild_option: Annotated[GuildConfigOption[int], ConfigMeta(default=100)] class ExampleCog(Cog): config: ExampleConfigSchema _config_identifier: int = 1234567890 # Consider using your Discord ID for this, instead of a random assortment of numbers. @override async def cog_load(self) -> None: await super().cog_load() # This is important! Tidegear has its own logic within the `cog_load` method that expects to be run. self.config = await ExampleConfigSchema.init(cog_name=self.metadata.name, identifier=self._config_identifier, logger=self.logger) @commands.is_owner() @commands.command() async def global_config(self, ctx: commands.Context, *, value: str | None = None) -> None: if value: # Works await self.config.example_global_option.set(value) # ValidationError: Incorrect type, expected `int` (got `list[int]`) await self.config.example_global_option.set([value]) await ctx.send(f"Set `{self.config.example_global_option.key}` to `{value}`") else: current_value = await self.config.example_global_option.get() # OR current_value = await self.config.example_global_option() # Both of these lines work and do the same thing. # The `__call__` interface is provided to be more familiar to Red Config users. reveal_type(current_value) # str await ctx.send(f"The current value of the `{self.config.example_global_option.key}` setting is `{current_value}`.") @commands.admin() @commands.guild_only() @commands.command() async def guild_config(self, ctx: commands.GuildContext, *, value: int | None = None) -> None: if value: await self.config.example_guild_option.set(ctx.guild, value) await ctx.send(f"Set `{self.config.example_guild_option.key}` to `{value}`") else: current_value = await self.config.example_guild_option.get(ctx.guild) reveal_type(current_value) # int await ctx.send(f"The current value of the `{self.config.example_guild_option.key}` setting is `{current_value}`.") ```

cswimr added the

enhancement

label 2025-11-10 15:29:51 -05:00

cswimr self-assigned this 2025-11-10 15:29:51 -05:00

cswimr added 1 commit 2025-11-13 10:07:51 -05:00

init config module ... 65bf9962b0

doesn't work yet but I want to get something committed

cswimr added 1 commit 2025-11-13 17:35:23 -05:00

add another paragraph to the module docstring 12abd4cb7f

cswimr added 1 commit 2025-11-13 17:40:54 -05:00

idiomatic -> declarative 64209aaab0

cswimr added 1 commit 2025-11-16 17:45:59 -05:00

only support pydantic BaseModel in Jsonable 4c935b8b57

cswimr added 11 commits 2025-11-24 07:29:17 -05:00

migrate metadata classes to pydantic models ae08dd9c89

give PartialGuild.in_guild a proper implementation 83216c3c45

simplify PartialGuild.in_guild implementation da20de8c73

use a raw discord import instead of importing symbols from discord ... 44940c5a5f

should help to make the code a bit more understandable, especially within Sentinel where you also have PartialUser/Guild/Role/Channel confusing things.

import cleanup 6e58ef4da5

add missing license header 5121b08ff8

change outdated docstring ... 22d8967b9a

as of tidegear 1.19.0, tidegear depends on pydantic directly. before that, it was an optional dependency, which was why this docstring recommended the use of `yarl`.

deprecate HttpUrl.yarl 4977d58256

oops 77205aab37

Update catthehacker/ubuntu:act-latest Docker digest to 2a8ed7d (#50) ... 342ed78cf4

Co-authored-by: Renovate <renovate@csw.im>
Co-committed-by: Renovate <renovate@csw.im>

Lock file maintenance (#51) ... 13a6412466

Co-authored-by: Renovate <renovate@csw.im>
Co-committed-by: Renovate <renovate@csw.im>

cswimr force-pushed feat/config/init from 13a6412466 to 4c935b8b57 2025-11-24 07:31:30 -05:00 Compare

cswimr added 1 commit 2025-11-24 07:33:35 -05:00

Merge branch 'main' into feat/config/init 7edc15eeb1

cswimr added 1 commit 2025-11-24 07:47:07 -05:00

fix tests not passing 9ce38a70c7

cswimr added 1 commit 2025-11-24 18:29:15 -05:00

move Jsonable to tidegear.config and redefine it; work on serialization 429cc2be9c

cswimr added 1 commit 2025-11-24 18:47:37 -05:00

Merge branch 'main' into feat/config/init 7aa8058da0

cswimr added 1 commit 2025-11-24 18:53:46 -05:00

correct error message 5a836c6a2d

cswimr added 1 commit 2025-11-24 18:55:01 -05:00

add copyright header 9178142819

cswimr added 1 commit 2025-11-24 20:35:56 -05:00

don't make ConfigSchema a pydantic class 9ce928530d

cswimr added 1 commit 2025-11-25 01:21:44 -05:00

add missing docstring 7de3ff7db2

cswimr added 1 commit 2025-11-25 09:49:07 -05:00

Merge branch 'main' into feat/config/init 29b0e0f486

cswimr added 1 commit 2025-11-25 09:49:58 -05:00

support normal pydantic basemodels in config instead of tidegear basemodels ee8756899c

cswimr added 1 commit 2025-11-25 09:55:07 -05:00

*actually* support normal pydantic basemodels 5354f56be8

cswimr added 1 commit 2025-11-25 09:57:26 -05:00

make a shallow copy before mutating dictionary 96e60c15b0

cswimr added 1 commit 2025-11-25 16:36:52 -05:00

implement proper serialization 0d3c7c24d3

cswimr added 1 commit 2025-11-25 16:46:15 -05:00

oops b39546642c

cswimr added 1 commit 2025-11-25 16:57:01 -05:00

adjust error message cd4d5f0ab1

cswimr added 1 commit 2025-11-25 17:10:42 -05:00

use the correct "cog" name for sentinel_config ce062a2cbc

cswimr added 2 commits 2025-11-25 17:35:32 -05:00

override init in SentinelConfigSchema 10a0aa13a4

support Mapping 3dddc4ae4b

cswimr added 1 commit 2025-11-25 17:35:55 -05:00

update incorrect docstring 5a571ee905

cswimr added 1 commit 2025-11-25 20:07:38 -05:00

support custom config options abea9ff82e

cswimr added 1 commit 2025-11-26 11:36:23 -05:00

refactor CustomConfigOption/Group, refactor sentinel type configuration f95501daa5

cswimr added 1 commit 2025-11-26 11:39:02 -05:00

update __init__.py 54de759bf9

cswimr added 1 commit 2025-11-26 12:43:45 -05:00

support more types; use pydantic for serialization fab9dce99b

cswimr added 1 commit 2025-11-26 14:08:00 -05:00

make tidegear.config its own full module instead of a single file 5e9427b4dd

cswimr added 1 commit 2025-11-26 14:09:39 -05:00

remove tidegear.config from tidegear's init d79e573347

cswimr added 1 commit 2025-11-26 14:10:14 -05:00

remove tidegear.config from tidegear's documentation e99a0d2069

cswimr added 1 commit 2025-11-26 14:27:52 -05:00

move ConfigMeta to schema & update module docstring ac425e55c7

cswimr added 1 commit 2025-11-27 02:07:29 -05:00

add some docstrings; import cleanup; and rename a kwarg parameter 739fd3992d

cswimr added 3 commits 2025-11-28 10:32:44 -05:00

clean up accepted types 2b060e0622

add .type() classmethod 4fec4e5e5d

move Sentinel's config schema to its own file 001629338c

cswimr added 1 commit 2025-11-28 10:58:53 -05:00

combine errors that occur during schema registration 6f7f67ae30

cswimr added 1 commit 2025-11-28 14:02:57 -05:00

add pyright comment 2c869c71b9

cswimr added 1 commit 2025-11-30 04:34:10 -05:00

implement tests 6d4e1b3702

cswimr added 1 commit 2025-11-30 04:46:13 -05:00

fix tests a6ce2a623c

cswimr added 1 commit 2025-11-30 04:49:01 -05:00

actually fix tests 320de2dc6c

cswimr added 1 commit 2025-11-30 04:52:20 -05:00

don't open in binary mode aefad0aad6

cswimr added 1 commit 2025-11-30 04:56:03 -05:00

fix pytest deprecation warning 2407a57048

cswimr added 1 commit 2025-11-30 11:05:10 -05:00

make tests pass on windows (in theory) 1526ea6f14

cswimr added 1 commit 2025-11-30 11:29:58 -05:00

add tests for GuildConfigOption 683b7f1e43

cswimr added 1 commit 2025-11-30 11:41:35 -05:00

rename a couple tests a23b2c1eb3

cswimr added 1 commit 2025-12-03 18:15:56 -05:00

do proper typechecking and validation on the version classvar 777d4ee81c

cswimr added 1 commit 2025-12-03 18:18:25 -05:00

simplify two conditionals d532d21f6f

cswimr added 1 commit 2025-12-03 22:41:40 -05:00

add migrations ba4378caeb

cswimr added 1 commit 2025-12-03 22:51:29 -05:00

catch when the user forgets to change config_version during migration e8d66312f3

cswimr added 1 commit 2025-12-06 12:47:03 -05:00

write a whole bunch of documentation and do some other stuff too i guess 42b1e99b79

cswimr added 1 commit 2025-12-06 12:53:55 -05:00

oops de9de9c904

cswimr added 1 commit 2025-12-06 17:58:28 -05:00

add missing docstring 200efe1537

cswimr added 1 commit 2025-12-06 18:32:00 -05:00

add BaseConfigSchema.options as a cached property 01cfbd6627

cswimr added 1 commit 2025-12-06 18:34:09 -05:00

rename options to registered_options to hopefully avoid namespace collision 97cc376e41

cswimr added 1 commit 2025-12-08 15:32:37 -05:00

enhance type safety; fix a serialization bug relating to pydantic models 97af1ab4eb

cswimr commented 2025-12-08 17:14:13 -05:00

Author

Owner

Copy link

I just want to get this merged, so configuration views / menus will come later. The core configuration system is done.

I just want to get this merged, so configuration views / menus will come later. The core configuration system is done.

cswimr added 1 commit 2025-12-08 17:14:27 -05:00

Merge branch 'main' into feat/config/init b7ff7189ef

cswimr changed title from WIP: Configuration system to Configuration system 2025-12-08 17:14:38 -05:00

cswimr scheduled this pull request to auto merge when all checks succeed 2025-12-08 17:14:57 -05:00

cswimr merged commit 2044b7edf1 into main 2025-12-08 17:15:23 -05:00

cswimr deleted branch feat/config/init 2025-12-08 17:15:23 -05:00

cswimr referenced this pull request from a commit 2025-12-08 17:15:24 -05:00

Configuration system (#49)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

help wanted

Need some help

  

invalid

Something is wrong

  

question

More information is needed

  

wontfix

This won't be fixed

No labels

bug

duplicate

enhancement

help wanted

invalid

question

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

cswimr

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

cswimr/tidegear!49

No description provided.