by obayesshelton on 4/22/25, 4:24 PM with 2 comments
I am trying to send about 5000 comments but I would love to send more but I am hitting limits of the message size.
What are some good design patterns when sending large sets of data to an LLM?
I ideally need to send all the data together at it gives context to the overall prompt.
by curious_curios on 4/22/25, 4:38 PM
- Group the comments by theme, then pass groups to the LLM to summarize/deduplicate then pass the outputs of that as context.
- RAG where only relevant parts are included in the context.
- Use an LLM with a larger context window (like Gemini pro)