I'm using llama-cpp-python==0.2.60, installed using this command CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python. I'm able to load a model using type_k=8 and type_v=8 (for q8_0 cache).

ORMs. Love 'em or hate 'em, they're often part of the job, and a core part of writing Django webapps. They can make it easy to write queries that work across databases, but the trade-off is your ...