RAG with ollama support#104
Conversation
|
I'll approve this one in about two weeks. I created a new tutorial of Ollama which I want to refer to |
|
can you please refer to https://github.com/NirDiamant/agents-towards-production/blob/main/tutorials/on-prem-llm-ollama/ollama_tutorial.ipynb in this tutorial? |
lordbeerus0505
force-pushed
the
rag-ollama
branch
from
1c84860 to
e3741e4
Compare
|
Apologies for the delay. Done ✅ |
|
Several things:
|
|
@lordbeerus0505 will you make the changes? |
lordbeerus0505
force-pushed
the
rag-ollama
branch
2 times, most recently
from
2a6076e to
4c38bc1
Compare
|
hey @lordbeerus0505 did you make the changes? Sorry for the super late response :) |
|
Hey, yes the changes were made. |
NirDiamant
left a comment
NirDiamant
left a comment
There was a problem hiding this comment.
Ollama support has been requested multiple times (#113). Thanks for working on this.
This PR has been open for a while - is it still up to date with the current main branch? If there are merge conflicts, a rebase would help move this forward.
… access to OpenAI keys, to try out RAGs locally for free.
lordbeerus0505
force-pushed
the
rag-ollama
branch
from
4c38bc1 to
4186540
Compare
📝 WalkthroughWalkthroughThe notebook adds an alternative local workflow for CSV-based Retrieval Augmented Generation using Ollama instead of OpenAI. Documentation and code cells demonstrate Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Ruff (0.15.9)all_rag_techniques/simple_csv_rag.ipynbUnexpected end of JSON input Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Done 👍 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@all_rag_techniques/simple_csv_rag.ipynb`:
- Around line 122-124: The notebook imports langchain_ollama (symbols:
langchain_ollama, ChatOllama, OllamaEmbeddings) but the install step omits that
package; update the environment setup/install cell to add the langchain-ollama
package so fresh runs don't raise ImportError before using Ollama models, then
re-run the install cell and confirm imports succeed.
- Around line 117-125: The notebook currently reassigns llm and vector_store in
multiple cells (e.g., ChatOpenAI then ChatOllama, FAISS with OpenAIEmbeddings
then overwritten by OllamaEmbeddings), causing silent provider overrides;
consolidate provider selection by reading a single switch (e.g., an ENV var like
PROVIDER) and use it to conditionally initialize only one stack: instantiate llm
as either ChatOpenAI or ChatOllama and vector_store as FAISS with matching
embeddings (OpenAIEmbeddings or OllamaEmbeddings) in one place (look for symbols
llm, vector_store, ChatOpenAI, ChatOllama, FAISS, OpenAIEmbeddings,
OllamaEmbeddings) so running all cells does not overwrite the chosen provider.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 1150d269-037e-400a-9604-95291c118ad2
📒 Files selected for processing (1)
all_rag_techniques/simple_csv_rag.ipynb
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from langchain_community.document_loaders.csv_loader import CSVLoader\n", | ||
| "from langchain_ollama import ChatOllama, OllamaEmbeddings\n", | ||
| "llm = ChatOllama(model=\"llama3.2\")" | ||
| ] |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify unconditional provider overwrites in notebook cell order.
python - <<'PY'
import json
p="all_rag_techniques/simple_csv_rag.ipynb"
nb=json.load(open(p))
for i,c in enumerate(nb["cells"]):
src="".join(c.get("source", []))
if "llm = ChatOpenAI" in src or "llm = ChatOllama" in src or "vector_store = FAISS(" in src:
print(f"cell[{i}] ->")
for line in src.splitlines():
if "llm = " in line or "vector_store = FAISS(" in line:
print(" ", line)
PYRepository: NirDiamant/RAG_Techniques
Length of output: 254
🏁 Script executed:
#!/bin/bash
# Extract cell boundaries and line numbers from the notebook
python - <<'PY'
import json
p="all_rag_techniques/simple_csv_rag.ipynb"
nb=json.load(open(p))
line_num = 1
for i, c in enumerate(nb["cells"]):
cell_start = line_num
src = "".join(c.get("source", []))
cell_lines = len(src.splitlines())
cell_end = line_num + cell_lines - 1
# Check for llm or vector_store assignments
has_llm_assign = "llm = ChatOpenAI" in src or "llm = ChatOllama" in src
has_vs_assign = "vector_store = FAISS(" in src
if has_llm_assign or has_vs_assign:
print(f"cell[{i}]: lines {cell_start}-{cell_end} ({cell_lines} lines)")
if has_llm_assign:
for line_idx, line in enumerate(src.splitlines(), start=cell_start):
if "llm = " in line:
print(f" Line {line_idx}: {line.strip()}")
if has_vs_assign:
for line_idx, line in enumerate(src.splitlines(), start=cell_start):
if "vector_store = FAISS(" in line:
print(f" Line {line_idx}: {line.strip()}")
line_num += cell_lines + 1 # Account for JSON structure
print("\n--- Checking line ranges mentioned in review ---")
# Check line 117-125 range
line_num = 1
for i, c in enumerate(nb["cells"]):
src = "".join(c.get("source", []))
cell_lines = len(src.splitlines())
line_start = line_num
line_end = line_num + cell_lines - 1
if (line_start <= 125 and line_end >= 117):
print(f"Lines 117-125 span cell[{i}] (lines {line_start}-{line_end})")
print(src[:300])
line_num += cell_lines + 1
# Check line range 376-385
line_num = 1
for i, c in enumerate(nb["cells"]):
src = "".join(c.get("source", []))
cell_lines = len(src.splitlines())
line_start = line_num
line_end = line_num + cell_lines - 1
if (line_start <= 385 and line_end >= 376):
print(f"Lines 376-385 span cell[{i}] (lines {line_start}-{line_end})")
print(src[:500])
line_num += cell_lines + 1
PYRepository: NirDiamant/RAG_Techniques
Length of output: 766
🏁 Script executed:
#!/bin/bash
# Get complete notebook structure and line counts
python - <<'PY'
import json
p="all_rag_techniques/simple_csv_rag.ipynb"
nb=json.load(open(p))
# Calculate total lines in notebook
total_lines = 1
for i, c in enumerate(nb["cells"]):
src = "".join(c.get("source", []))
lines = len(src.splitlines())
total_lines += lines + 1
print(f"Total notebook lines: ~{total_lines}")
print(f"Total cells: {len(nb['cells'])}\n")
# Print all cells with llm/vector_store assignments and their full content
line_num = 1
for i, c in enumerate(nb["cells"]):
src = "".join(c.get("source", []))
cell_lines = len(src.splitlines())
cell_start = line_num
cell_end = line_num + cell_lines - 1
has_llm = "llm = " in src
has_vs = "vector_store = " in src
if has_llm or has_vs:
print(f"\n=== cell[{i}] (lines {cell_start}-{cell_end}) ===")
print(src)
line_num += cell_lines + 1
PYRepository: NirDiamant/RAG_Techniques
Length of output: 1729
Consolidate provider selection to avoid silent overrides from sequential cell execution.
The notebook unconditionally overwrites llm and vector_store across multiple cells: cell[5] initializes ChatOpenAI, but cell[7] unconditionally reinitializes llm to ChatOllama; similarly, cell[14] sets up FAISS with OpenAI embeddings, but cell[16] unconditionally overwrites it with Ollama embeddings. Running all cells shifts the entire stack to Ollama regardless of the initial provider choice. Use a single provider switch (e.g., environment variable) and initialize one consistent stack per notebook run.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@all_rag_techniques/simple_csv_rag.ipynb` around lines 117 - 125, The notebook
currently reassigns llm and vector_store in multiple cells (e.g., ChatOpenAI
then ChatOllama, FAISS with OpenAIEmbeddings then overwritten by
OllamaEmbeddings), causing silent provider overrides; consolidate provider
selection by reading a single switch (e.g., an ENV var like PROVIDER) and use it
to conditionally initialize only one stack: instantiate llm as either ChatOpenAI
or ChatOllama and vector_store as FAISS with matching embeddings
(OpenAIEmbeddings or OllamaEmbeddings) in one place (look for symbols llm,
vector_store, ChatOpenAI, ChatOllama, FAISS, OpenAIEmbeddings, OllamaEmbeddings)
so running all cells does not overwrite the chosen provider.
| "from langchain_community.document_loaders.csv_loader import CSVLoader\n", | ||
| "from langchain_ollama import ChatOllama, OllamaEmbeddings\n", | ||
| "llm = ChatOllama(model=\"llama3.2\")" |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify notebook imports vs install command dependency list.
python - <<'PY'
import json, re
p="all_rag_techniques/simple_csv_rag.ipynb"
nb=json.load(open(p))
install_cells=[]
imports_ollama=False
for i,c in enumerate(nb["cells"]):
src="".join(c.get("source", []))
if "pip install" in src:
install_cells.append((i, src.strip()))
if re.search(r'from\s+langchain_ollama\s+import', src):
imports_ollama=True
print("imports_langchain_ollama:", imports_ollama)
print("install_cells:")
for i,s in install_cells:
print(f" cell[{i}]: {s}")
print("has_langchain_ollama_in_install:",
any("langchain-ollama" in s for _,s in install_cells))
PYRepository: NirDiamant/RAG_Techniques
Length of output: 283
Add missing langchain-ollama dependency to install step.
Line 123 imports langchain_ollama, but the install cell currently omits langchain-ollama. Fresh runs will fail with an ImportError before the Ollama models are usable.
Suggested fix
- !pip install faiss-cpu langchain langchain-community langchain-openai pandas python-dotenv
+ !pip install faiss-cpu langchain langchain-community langchain-openai langchain-ollama pandas python-dotenv🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@all_rag_techniques/simple_csv_rag.ipynb` around lines 122 - 124, The notebook
imports langchain_ollama (symbols: langchain_ollama, ChatOllama,
OllamaEmbeddings) but the install step omits that package; update the
environment setup/install cell to add the langchain-ollama package so fresh runs
don't raise ImportError before using Ollama models, then re-run the install cell
and confirm imports succeed.
2 participants
I assume multiple readers of this repository such as myself do not have access to OpenAI, Groq or Anthropic's API keys and don't want to spend money on them. Further, many organizations, such as the one I work in, would prefer our models do not share our proprietary data to OpenAI. A local model would come in clutch in such scenarios.
Has been tested on my macos 15 (M1 Macbook Air) laptop.
Summary by CodeRabbit
New Features
Documentation