How to change model for indexing?

[arash-ashra]

gpt3 sucks at math and code! I'm trying to use gpt4 for indexing but with no luck. It'd be great if there was a model parameter for indexing commands. currently, it only supports while querying which is not helpful if the context is written using gpt3.

I also tried setting the settings parameter model to gpt-4 but it didn't seem to work.

[Kav-K]

What do you mean? Are you on the latest version of the bot?

The bot supports models as follows in /index chat:

as you can see there's a model param to choose what u want to use for the indexed conversation

During query-time you can also set the model:

For actual indexing itself, the LLM isn't used, indexing actually just consists of chunking input text and then creating embeddings. LLMs are only used when querying a created index, or when creating a deep composition index.

@ashra-academy does that help

[arash-ashra]

Rightt I meant during deep composition. is it possible to set the model during deep composition?

[arash-ashra]

also which model is used for embeddings? is it an openai ada? being able to set the embedding model would solve the issue too I think if the model is trained more on math and code like gpt4

[Kav-K]

Oh I see, unfortunately not right now, it is editable in code but not within the discord bot while it's in use AFAIK, what model do you want to switch it to? I can look into it and fix the default to be some sort of gpt-4 model.

For the embeddings model it is the latest ada model yeah, but I'm not aware of any other embedding systems that are trained more on math and code, gpt-4's embeddings would also use ada

[Kav-K]

For future note for work on this ticket:

• We need to fix the passing of the llm predictor into the deep composer, right now it is using some sort of default llm predictor and not the gpt-4-32k one defined in the compose_indexes method

[arash-ashra]

there are alot of embedding models available, the code models would be good at math as well. so I'd propose we be able to choose them with a parameter. here's a discussion: https://community.openai.com/t/what-do-all-these-models-do/19007/5

[Kav-K]

@ashra-academy that's not true, the only embedding models usable are:

And the only GOOD one is the last most recent one, so we don't have openai options for embedding models that work better with code and text.

[Kav-K]

All of those "code" models you're talking about are also LLMs, not embedding models, and are incredibly old and much worse compared to GPT-4. Out of all of the models listed, gpt-4-1106-preview performs the best across the board

[arash-ashra]

it looks like available on my dashboard. probably the highlighted one would be best for math as its a mix of latex and text.

[Kav-K]

Oh you're totally right, that's my bad!

I see it in my dashboard as well, I don't see official documentation for any of these models though so I need to do some digging to figure out how these are used and if they're beter than gpt-4, and how they actually even work

[arash-ashra]

Awesome! looking forward to trying them out. Cheers!

[arash-ashra]

my understanding is that all the Text models are embedding models

[Kav-K]

Thanks for bringing this to my attention! Let me know if you come across any good docs for these

[arash-ashra]

Sure thing