My biggest takeaway here is that choosing the context length and (to a lesser extent) the temperature carefully is important for reducing hallucinations. I expected model families to vary widely between themselves but not for context length to have such a massive impact tbh.

It seems from this like reducing context length in applications where it isn’t essential for the model to hold very large amounts of context simultaneously would be best practice no?

And diesel prices (which underpin essentially all industry and production) have gone even more brrrrrrrrrr. Which means just about everything else will be shooting up in price soon if the strait remains closed…