Milla is an IRC bot that:
- sends things over to an LLM when you ask it questions and prints the answer with optional syntax-highlighting.
Currently supported providers:
- Ollama
- Openai
- Gemini
- Milla can run more than one instance of itself
- Each instance can connect to a different ircd, and will get the full set of configs, e.g. different proxies, different postgres instance, ...
- You can define custom commands in the form of SQL queries to the database with the SQL query result being passed to the bot along with the given prompt and an optional limit so you don't go bankrupt(unless you are running ollama locally like the smart cookie that you are).
milla accepts one cli arg which tells it where to look for the config file:
Usage of ./milla:
-config string
path to the config file (default "./config.toml")
The bot will respond to chat prompts if they begin with botnick:
.
The bot will see a chat prompt as a command if the message begins with botnick: /
.
An example is provided under config-example.toml
. Please note that all the config options are specific to one instance which is defined by ircd.nameofyourinstance
.
The address for the IRC server to connect to.
The nick the bot should use.
Whether to use SASL for authentication.
The SASL username.
The SASL password for SASL plain authentication. Can also be passed as and environment variable.
The address for the Ollama chat endpoint.
The name of the model to use.
The style to use for syntax highlighting done by chroma. This is basically what's called a "theme".
The formatter to use. This tells chroma how to generate the color in the output. The supported options are:
noop
for no syntax highlightingterminal
for 8-color terminalsterminal8
for 8-color terminalsterminal16
for 16-color terminalsterminal256
for 256-color terminalsterminal16m
for treucolor terminalshtml
for HTML output
NOTE: please note that the terminal formatters will increase the size of the IRC event. Depending on the IRC server, this may or may not be a problem.
Which LLM provider to use. The supported options are:
- ollama
- chatgpt
- gemini
The apikey to use for the LLM provider. Can also be passed as and environment variable.
The system message to use for ollama.
The path to the client certificate to use for client cert authentication.
The password to use for the IRC server the bot is trying to connect to if the server has a password. Can also be passed as and environment variable.
Which address to bind to for the IRC server.
The temperature to config the model with.
The timeout for requests made to the LLM provider.
How much to wait before reconnecting to the IRC server.
Which port to connect to for the IRC server.
How many conversations to keep in memory for a model.
Ping delay for the IRC server.
Ping timeout for the IRC server.
Skip verifying the IRC server's TLS certificate. This only makes sense if you are trying to connect to an IRC server with a self-signed certificate.
Whether to use TLS to connect to the IRC server. This option is provided to support usage on overlay networks such as Tor, i2p and yggdrassil.
Disables the "fallback" to a non-TLS connection if the strict transport policy expires and the first attempt to reconnect back to the TLS version fails.
Disable girc's built-in flood protection.
Whether to enable debug logging. The logs are written to stdout.
Whether to write raw messages to stdout.
List of admins for the bot. Only admins can use commands.
admins = ["admin1", "admin2"]
List of channels for the bot to join when it connects to the server.
ircChannels = ["#channel1", "#channel2"]
Please note that the bot does not have to join a channel to be usable. One can simply query the bot directly as well.
Name of the database user.
Password for the database user.
Address of the database.
Name of the database.
Determines which proxy to use to connect to the IRC network:
ircProxy = "socks5://127.0.0.1:9050"
Determines which proxy to use to connect to the LLM endpoint:
llmProxy = "socks5://127.0.0.1:9050"
Name of the milla instance, must be unique across all instances.
Milla will only answer if the nick is in the admin list.
webirc gateway to use.
webirc hostname to use.
webirc password to use.
webirc address to use.
Custom commands let you define a command that does a SQL query to the database and performs the given task. Here's an example:
[ircd.devinet_terra.customCommands.digest]
sql = "select log from liberanet_milla_us_market_news;"
limit = 10
prompt = "give me digest of the provided news"
[ircd.devinet_terra.customCommands.summarize]
sql= "select log from liberanet_milla_us_market_news;"
limit= 300
prompt= "given all the data, summarize the news for me"
In the above example digest and summarize will be the names of the commands: milla: /cmd summarize
.
Currently you should only ask for the log column in the query. Asking for the other column will result in the query not succeeding.
The limit
parameter limits the number of SQL queries that are used to generate the response. Whether you hit the token limit of the provider you use and the cost is something you should be aware of.
A limit
value of 0 disables the limit on the amount of rows that are passed to milla.
NOTE: since each milla instance can have its own database, all instances might not necessarily have access to all the data milla is gathering. If you use the same database for all the instances, all instances will have access to all the gathered data.
[ircd.devinet]
ircServer = "irc.myawesomeircnet.com"
ircPort = 6697
ircNick = "milla"
enableSasl = true
ircSaslUser = "milla"
ircSaslPass = "xxxxx"
ircChannels = ["##chan1", "##chan2"]
temp = 0.2
requestTimeout = 10
millaReconnectDelay = 60
model = "gpt-3.5-turbo"
chromaStyle = "rose-pine-moon"
chromaFormatter = "terminal256"
provider = "chatgpt"
apikey = "xxxx"
memoryLimit = 20
admins = ["noone_has_this_nick"]
debug = true
out = true
databaseAddress = "postgres:5432"
databasePassword = "changeme"
databaseUser = "devi"
databaseName = "milla"
scrapeChannels = ["#soulhack", "#warroom"]
ircProxy = "socks5://127.0.0.1:9050"
llmProxy = "http://127.0.0.1:8180"
skipTLSVerify = false
useTLS = true
[ircd.liberanet]
ircServer = "irc.libera.chat"
ircNick = "milla"
model = "gpt-3.5-turbo"
ircPort = 6697
chromaStyle = "rose-pine-moon"
chromaFormatter = "terminal16m"
provider = "gemini"
apikey = "xxxx"
temp = 0.5
requestTimeout = 10
millaReconnectDelay = 60
keepAlive = 20
memoryLimit = 20
pingDelay = 20
pingTimeout = 600
skipTLSVerify = false
useTLS = true
disableSTSFallback = true
allowFlood = false
admins = ["noone_has_this_nick"]
ircChannels = ["##milla1", "##milla2"]
debug = true
out = true
ircProxy = "socks5://127.0.0.1:9051"
llmProxy = "http://127.0.0.1:8181"
[ircd.liberanet.customCommands.digest]
sql = "select log from liberanet_milla_us_market_news;"
limit = 10
prompt = "give me digest of the provided news"
[ircd.liberanet.customCommands.summarize]
sql= "select log from liberanet_milla_us_market_news;"
limit= 300
prompt= "given all the data, summarize the news for me"
Prints the help message.
Get the value of a config option. Use the same name as the config file but capitalized: /get chromaFormatter
Get the value of all config options.
Set a config option on the fly. Use the same name as the config file but capitalized: /set chromaFormatter noop
Returns memory stats for milla.
Joins a channel: /join #channel
Leaves a channel: /leave #channel
Images are automatically pushed to dockerhub. So you can get it from there.
An example docker compose file is provided in the repo under docker-compose.yaml
.
milla can be used with gvisor's docker runtime, runsc
.
services:
milla:
image: milla
build:
context: .
deploy:
resources:
limits:
memory: 64M
logging:
driver: "json-file"
options:
max-size: "100m"
networks:
- millanet
user: ${UID}:${GID}
restart: unless-stopped
command: ["--config", "/opt/milla/config.toml"]
volumes:
- ./config-gpt.toml:/opt/milla/config.toml
- /etc/localtime:/etc/localtime:ro
- /etc/resolv.conf:/etc/resolv.conf:ro
cap_drop:
- ALL
runtime: runsc
networks:
millanet:
driver: bridge
milla can be configured to store all incoming public messages for future use in a postgres database. An example docker compose file is provided under docker-compose-postgres.yaml
.
services:
terra:
image: milla_distroless_vendored
build:
context: .
dockerfile: ./Dockerfile_distroless_vendored
deploy:
resources:
limits:
memory: 128M
logging:
driver: "json-file"
options:
max-size: "100m"
networks:
- terranet
user: 1000:1000
restart: unless-stopped
entrypoint: ["/usr/bin/milla"]
command: ["--config", "/config.toml"]
volumes:
- ./config-gpt.toml:/config.toml
- /etc/localtime:/etc/localtime:ro
cap_drop:
- ALL
environment:
- HTTPS_PROXY=http://172.17.0.1:8120
- https_proxy=http://172.17.0.1:8120
- HTTP_PROXY=http://172.17.0.1:8120
- http_proxy=http://172.17.0.1:8120
postgres:
image: postgres:16-alpine3.19
deploy:
resources:
limits:
memory: 4096M
logging:
driver: "json-file"
options:
max-size: "200m"
restart: unless-stopped
ports:
- "127.0.0.1:5455:5432/tcp"
volumes:
- terra_postgres_vault:/var/lib/postgresql/data
- ./scripts/:/docker-entrypoint-initdb.d/:ro
environment:
- POSTGRES_PASSWORD_FILE=/run/secrets/pg_pass_secret
- POSTGRES_USER_FILE=/run/secrets/pg_user_secret
- POSTGRES_INITDB_ARGS_FILE=/run/secrets/pg_initdb_args_secret
- POSTGRES_DB_FILE=/run/secrets/pg_db_secret
networks:
- terranet
- dbnet
secrets:
- pg_pass_secret
- pg_user_secret
- pg_initdb_args_secret
- pg_db_secret
runtime: runsc
pgadmin:
image: dpage/pgadmin4:8.6
deploy:
resources:
limits:
memory: 1024M
logging:
driver: "json-file"
options:
max-size: "100m"
environment:
- PGADMIN_LISTEN_PORT=${PGADMIN_LISTEN_PORT:-5050}
- PGADMIN_DEFAULT_EMAIL=${PGADMIN_DEFAULT_EMAIL:-devi@terminaldweller.com}
- PGADMIN_DEFAULT_PASSWORD_FILE=/run/secrets/pgadmin_pass
- PGADMIN_DISABLE_POSTFIX=${PGADMIN_DISABLE_POSTFIX:-YES}
ports:
- "127.0.0.1:5050:5050/tcp"
restart: unless-stopped
volumes:
- terra_pgadmin_vault:/var/lib/pgadmin
networks:
- dbnet
secrets:
- pgadmin_pass
networks:
terranet:
driver: bridge
dbnet:
volumes:
terra_postgres_vault:
terra_pgadmin_vault:
secrets:
pg_pass_secret:
file: ./pg/pg_pass_secret
pg_user_secret:
file: ./pg/pg_user_secret
pg_initdb_args_secret:
file: ./pg/pg_initdb_args_secret
pg_db_secret:
file: ./pg/pg_db_secret
pgadmin_pass:
file: ./pgadmin/pgadmin_pass
The env vars UID
and GID
need to be defined or they can replaces by your host user's uid and gid.
As a convenience, there is a a distroless dockerfile, Dockerfile_distroless
also provided.
A vendored build of milla is available by first running go mod vendor
and then using the provided dockerfile, Dockerfile_distroless_vendored
.
For a regular build:
go mod download
go build
For a vendored build:
go mod vendor
go build
- I end up with color escape sequences getting printed at the end of a line/begging of the next line. What gives?
This is happening because you have reached the message limit on irc which 512 for the event. This practically leaves around 390-400 character left for the message itself. Certain ircds allow for bigger sizes and certain clients might do. But most ircds dont send
linelen
to the clients. In a closed-loop situation where you control everything, as in, the ircd and all the clients(i.e. A private irc network), you can try to increase thelinelen
for the ircd and the client. Please note that the client in this case is girc. You irc client can have its own set of limits too. The 512 limit is hardcoded in girc. You can vendor the build or use the vendored dockerfile, change the hard limit and run milla with an increased limit. Needless to say, you can try to use a chromaFormatter that produces less characters which is basically not using truecolor orterminal16m
.
Milla would not exist without the following projects:
- plugins support