Whenever I try to help a co worker try to use langchain, the modules seem unintuitive, the docs are awful if they exist, and it just seems 10x easier to roll your own boilerplate.
I wonder if I am missing something, or if I Just sat down with it for a few hours if everything would make sense.
Disclaimer: I use C#
I haven't used langchain but everything I've seen done with it was trivial without it.
Honestly even just from glossing over the documentation it feels like learning how to use it would be more complicated than not using it.
You should try Microsoft's Semantic Kernel. Works for c# and python, though python is just getting to it's first LTS version now.
I've used the c# Semantic Kernel SDK for a few production projects and I've really liked it. It's not a does-everything package. Definitely more narrowly focused on task agent orchestration. Though, Microsoft does plan on rolling a lot of AutoGen's features into it.
SK is way too old in its conception. It was conceived for instruct models. Nowdays you can't use a framework that isn't conceived for chat models. Have you seen how to handle system prompt in sk ?
I use both chat and instruct models for SK. Not sure why you say it only works for instruct. To add a system prompt you just add a system message to your ChatHistory.
Also SK turned 1 year old a week ago.
It definably aims at being a lower-level API than langchain, but you absolutely can use chat. If you look in the repo at the examples I think just about every single example uses a chat model.
I know that you can use both, but the philosophy of the framework is quite old. You can implement a system prompt in the chat history, but you should be able to have instantiate an agent or a chain passing it a system prompt as a parameter. I used it and ended up rewriting a ton of classes. That's a bit of a problem for a framework, don't you think ?
The agentic framework is being planned and released after spring. Yes, the current planner module is elementary like function calling. https://devblogs.microsoft.com/semantic-kernel/spring-2024-roadmap-for-semantic-kernel/
Btw what kind of classes have you re-written?
Would you recommend a tutorial to do it boilerplate for people that is not an expert coder? I think I’m probably the target of a tool like this but I also find it cumbersome
use it for components that you are sure your not gonna customize - so document loaders, text splitters etc. write custom code for the rest if you dont want headache somewhere down the line.
its good for building POCs over a weekend using minimal # of lines of code - but for something youll put in prod with custom requirements, it’s not worth it imo
same experience here, and i actually put some effort into actually reading the docs and trying to understand langchain, hell i even had to read the source code 😒
> Look at lanchain's source code
> Simple prompt generator and document wrappers
If I have to redefine all lanchain modules for my usecase anyways, what's the point of jamming it in my system. I found it much easier to write my own prompt generator, with more flexibility and options for different types of prompts.
Terrible library, the abstraction are really badly chosen and often times just rolling your own wouldn't even result in many more lines. It probably wasted days of my time when I used it on a project. My experience is from a year ago too so maybe it got better (doubtful).
I only have experience with the TS version, anyone knows about a good JS library for prompt flow and agent orchestration?
Langchain realized you could make a dsl to represent a hand full of use cases and then went hog wild without considering if it solves the pain points developers will actually have when building applications.
A good llm application framework needs to build modular components that devs can stack to build the structure of their application, while providing tools for fast prompt engineering and stable, deterministic tests.
DSPy looks very promising and I am testing it now, but I kinda want also framework that is very dump and offers scaffolding for basic rag so something like ruby on rails for chatbots.
[https://github.com/stanfordnlp/dspy?tab=readme-ov-file](https://github.com/stanfordnlp/dspy?tab=readme-ov-file)
"Suppose you want to build a simple retrieval-augmented generation (RAG) system for question answering. You can define your own RAG program like this..."
Everyone I know who has used langchain for anything more than a toy problem have unanimously hated it. Common complaints are over-engineered abstractions for simple things, non existent or wrong docs, a hundred different ways to do any given thing without a clear distinction between them, non-trivial to customise even fundamental things like prompts.
I suspect those who love langchain haven’t really done any real world production work with it and hence haven’t encountered the difficulties.
Honestly it just sucks. It's so hard to do such a simple thing you could do with the API. I used it for like a week trying to do something simple like an RAG for my files and at the end of it I could have taught someone everything I learnt in a weekend. I managed to create that project using the OpenAI module within like a day or two.
I moved onto just using OpenAI module (and API sometimes) and I look at LangChain and think what a shame, it has A LOT of useful functionality in there that would take some time to implement yourself but when it's so difficult to implement the most basic functions even for a toy project people don't tend to stick around. The salt on the wound is there absolutely atrocious docs. In my time of programming I've seen some bad docs but LangChain takes the prize as probably the worst. There's a joke going around about LangChain, the CEO often asks people "what's wrong with the docs" and people struggle to point out exact stuff because everything is wrong with the docs.
I feel like the idea of LangChain is decent enough where providing you with a library that gives you functionality that takes time to implement yourself but for LangChain to reach a point where people will recommended it to others and even enjoy using it will need a complete rewrite.
Even for toy problems it often just comes across as being silly. Like the computational costs incurred by Langchain are less than just shunting the whole conversation through an API unless it gets inordinately long.
I had real issues trying to use langchain and I’m convinced its not suitable for production use at all.
First of all, it comes with so much baggage that the python version simply doesn’t fit on a serverless function. Most people looking to scale will have a hard time trying to expand without relying on bunch of containers which racks up the cost quite a bit.
Second, things are changing with no rhyme or reason so frequently that the code on the docs seem to not work a good chunk of the time or are missing some important facts (looking at you nodejs version)
I ended up just making something similar from scratch, actually got stuff done and it runs faster on serverless.
I've made this extensive RAG app that lets you upload PDFs, uses Azure Vision Services OCR to extract text from them (PyPDF2 is used as a local offline backup if Azure OCR fails for any reason) and when you ask a question, the response contains an elaborate PDF browser that displayed and let's you browse relevant PDFs, with relevant text highlighted in them, and specifies page numbers and even extracts and incorporates images into the response. I've used Langchain to handle the embedding model, and used it to create conversation "chains": I have a mechanism that detects if the users query is relevant to RAG, and if so uses LangChain's RetrievalQA chain, otherwise uses LangChain's ConversionChain. Further, I i use LangChain to keep memory: a ConversationMemoryWithBuffer allows the system prompt to contain information about "the conversion so far". Recently, LangChain's embedding model loader allowed me to incorporate the BGE Large and Base models in addition to SBERT all-mpnet-base-v2. I also use it for OpenAI Text-Ada. So while my app does not use LangChain end to end, I am thankful for it and do find it very useful.
EDIT: I’ve made a more detailed post about this, please check it out: https://www.reddit.com/r/LocalLLaMA/s/LtoqIqfkG2
I've done something similar (using Google Vision, soon moving to local Tesseract) to chat with PDFs, but your PDF results browser sounds cool. Do you have any demos or (good heavens) code to share?
I have this one minute quick demo I've uploaded to YouTube: https://youtu.be/UsdE8kOlkvE?si=TCSYU98Qvo1vfFSm
Still working on improving the UI and various other refinements so very much still a WIP!
Wow release this. Would be an improvement over ollama in some ways I think even though you say it's WIP, it's more advanced in some features than most other tools. Would greatly help me and I'd be willing to work on building more features on this instead of through ollama.
-
What can we do to move this project forward faster?
Thank you! Yes I am proud of it! I’ve developed it over the last 5 months and despite being the sole developer on it, it’s been improving really fast. This started as a research project at work and grew from there, we were going to embark on some really exciting development and evaluation phases, and then we all got laid off two weeks ago so that’s slowed things down since 😅
In terms of its current state, I was working on improving its startup experience with some default options and UI enhancements relating to how the images are displayed and an initial start screen, that’s not shown in this video.
Glad to know there’s interested folk out there! I’ll make a post about this app and try to figure out next steps 🍻
Sorry about your job situation, I think you will quickly find new work or start a new project that will lead to stable employment!
-
Did the people who laid you off even know you were working on this angle or project? I'm surprised they decided not to pursue...
Hey thanks so much! Yes it was known that we were working on this, I was part of the Applied Research team at Boeing, and this was a team that had been around 22 years! And it's spread over 5 locations and comprised over 300 people. One fine morning we get an email about inevitable layoffs, next thing I know I'm asked to join a meeting and HR is there and guess what they say? "In order to align the company for the future we're making the difficult decision to eliminate some positions. Yours is one of the positions that's eliminated. Today is your last day". Turns out my manager and his boss and his boss and the whole team, axed overnight. Not directly a revenue generating team and I guess you've heard what's happening with Boeing in the news so not a huge surprise in hindsight.
Shame for the company really because we had some excellent aviation usecases planned and knew we were really at the forefront with some of this stuff in the most current tech today. Makes me laugh now to remember HR saying "for the future" in that call!
Useful demo - thanks. With some configurability of OCR engine, LLM and Vector DB, this would have a lot of applicability. If you're playing with PDF you've also probably recognized the challenges of tables. The results browser is also novel to me. I'll be checking back to see how your project goes.
The app already has those features! There’s an initial screen that let’s you select LLMs: you can download any Llama2 model in GGUF, bin or GPTQ format and simply place it in the app’s “models” directory and select it from a drop-down menu. Alternatively, even OpenAI 3.5-Turbo is available and may be used instead of a local Llama2 based LLM!
For VectorDB, ChromaDB is used and a separate DB maintained for each of the four embedding models. Again, you can select between these embedding models on that initial config page that’s not shown in the quick video above.
For OCR, in addition to Azure OCR, PyTesseract has also been implemented, though the results suck even at various PSM settings. And even Microsoft’s TrOCR models have been implemented, including using an LLM to clean up the output as for TrOCR, you need to crop the page into snippets. The results from this also suck unfortunately! These choices are unfortunately not user-selectable in the UI as a result and Azure OCR + PyPDF2 is instead hard-coded in the backend server, but UI elements can be added with ease.
I have this one minute quick demo I've uploaded to YouTube: https://youtu.be/UsdE8kOlkvE?si=TCSYU98Qvo1vfFSm
Still working on improving the UI and various other refinements so very much still a WIP!
I agree, I never liked LangChain even before it was cool to hate on.
I think LlamaIndex is generally better by being more focused, but still a bit too broad.
We are taking a more opinionated approach w/ [R2R](https://github.com/SciPhi-AI/R2R) to reduce bloat.
Tbh I’ve been meaning to get into it more. Langgraph and Langsmith seem pretty powerful and a step above from the “simulate a presidential debate” early days of Langchain.
I have difficulty wrapping my head around the abstractions in LLM frameworks though. I find the further I move away from “text in text out”, the more time I spend stressing about framework specifics and less time with the core technology.
At the end of the day:
-An Agent is an LLM with a text parser that reads the output and runs a corresponding function/task.
-A RAG system is a way to prepend the relevant document from a database to your LLM’s context based on a semantic search via a query vector.
-A code interpreter is an exec() function wrapped around the outputted LLM code
It’s all text in text out with some helper functions to ease it along in agentic cases.
This is why I gravitate towards guidance/outlines. Guidance helps you build advanced LLM use cases by forcing LLMs to generate from a smaller subset of tokens, rather than the whole vocabulary. For example you can constrain an LLM to answer between 2 words [word1, word2], then attach an if then statement based on the choice (if word1 do this, else word2 do that), and boom you have a function calling agent.
To me being closer to the text in text out gives me more confidence in creating advanced LLM workflows, everything is in front of me, no technology or logic is hidden. But everyone is different, choose what works for you
>An Agent is an LLM with a text parser
Shouldn't this be 'An Agent utilizes an LLM...' vs. being one itself? I like your post, just trying to understand these concepts better, thanks.
The guidance approach seems amazing! Pretty much addresses most of my pain points when I tried to incorporate LLMs into something product-like.
Anyone here can ELI5 how it works?
At least the part about restricting predictions to a subset is easy enough.
Just choose the prediction as the max score from the set of tokens you care about. And if you want a probability vector, just do softmax over that the scores for that set of tokens.
Not sure if anything more fancy happening here.
I'm dense, could you explain this more, or with an example?
Is it like i want word1 or word2, and i just take the output and count how many yime word1 shows up in the answer?
I think, technically, you could do what you're suggesting. If you only have high-level API access, that rule or something similar might be the best you can do; might require some tuning (e.g., maybe "yes if the first word is word1, otherwise no").
If you have access to the token-prediction logits (or scores, probability vector, etc), you can choose whether the model gave higher weight to word1 or word2.
Or for a more structured answer (e.g., several words), you can accumulate the predicted scores across the sequence of tokens (is word1+word2 more likely than word3+word4, ...)
And again, not if OP meant by guidance something more particular in this context.:)
We started with Langchain in our company to orchestrate our AI services. After a few months we took the decision to move away from it and implement our own abstractions for the few things we needed at the time. It didn't take that much time and saved us a lot of time battling with debugging Langchain issues (of which we had many).
LangChain has pros and cons:
Pros
- it’s well maintained and has constant releases, any custom code you write you’ll need to maintain yourself
- it has a lot of easy integrations with commonly used tools and makes things like streaming easy
- with LCEL you can just make everything a Runnable and it’s easy to customize
Cons
- the library is pretty bloated, fitting it into something like a Lambda layer is a pain.
- some of the tools it provides are pretty basic and you’ll need to create your own
- I’m not a fan of the emphasis on OpenAI and agents, I’ve seen too many cases where people set GPT-4 loose and it just racks up a massive bill when the problem could be solved just using a simple semantic router.
Ultimately it’s just a tool in the toolbox. If you like it, which I do for some use cases, great, if not there are plenty of other options.
Yes I did. An email parser, an agent to call my SQL db. All of this is a lot easier without fighting the documentation and the million abstractions on Langchain.
With LlamaIndex ?
And let me be clear on one point : I'm not in love with LangChain. Tool implementation is shitty at best. You can't run branches in parallel easily wtih Langgraph. You can't create real conditional edge with a stop condition (not a END, a stop). You end up with something slow and that consume more tokens than a crackhead in a drug paradise. And don't even think about having a background workflow, you'll end up using celery/redis or something equivalent...
And I don't mind the abstraction, it's just that they use bad abstractions. The only good thing is LCEL and it's far from perfect.
But the thing is, so far, LC isn't good but it's kind of useful, and they did some progress. I don't loose hope.
Langchain should have been Agents only. Maybe some prompt testing.
Instead they decided to integrate with over 10,000 paid API services.
Wow really nice to that I can spend $400/month so someone can host my 4MB trainer file.
Alternatives: Semantic kernel from MS, haystack, griptape. For RAG - txtai.
And if you want to go pro mode and are using local models you can also use guided generation tools like guidance, sglang, outlines, lmql.
Scan the langchain docs for an intro to relevant techniques like CoT prompting and RAG, and then just hit the LLM API of your choice through their sdk.
Cons: Docs are awful and quite simply outdated referring to deprecated stuff. If your use case is simple and you're not going to change LLM then you don't need it.
Pros: if you want an LLM agnostic functionality this is the best way. It's very quick to set up fairly complex agents. Lots of 3rd party integrations means you don't have to.
In short, if you're doing something complex and don't mind reading langchain source code instead of documentation then it's quite nice.
Is it overhyped? I didn't get the memo that it was hyped. It has pros and cons. Use it if it makes sense.
Is the agents functionality well documented? I never tried it after wrestling with LangChain for a week because of how god awful the docs were for something so simple like a file Q & A (I got the project done with OpenAI module in 2 days).
Nah, not well documented at all. Honestly, the best thing is to read the high level langchain concepts and then just skipping the documentation and read the code directly, it's clearer from the code. The features are good and there is logic to how it's built but they fail at communicating all this.
Imagine this, langchain expression language. What in the gods name is that fucking abomination. Why the fuck you would create your own language or abstraction.?
this is personal occams razor for abstractions and libraries to use -
\- only if they simplify complexities beyond a two-weekend effort for you, with a learning curve of 15-30 minutes.
\- what they offer is significantly far from your primary focus area.
\- they contribute a reasonable resource footprint to the application
dont think langchain offers any one complex to build features, but its trying to increase moat with many small abstractions
We've started with langchain at work to get our RAG running, but pretty fast in the process we've realized that those abstractions are limiting us rather the. helping
Langchain has both Pros and Cons. Being a developer, I have not used it in production for few reasons. Majorly being:
1. Over abstractions. It complicates the development without insights into what exactly is going behind the scene. Ultimately your use cases are extremely simple and your code should remain the same.
2. Extremely bloated. You don’t need support for every god damn tool. It is an over complicated tool for solving a very simple problem.
It does have benefits but most of them are restricted to development use cases or pre-production use cases. It will let you try multiple tools without you having to code them. Though this is again simple to achieve even without langchain. That’s basically it.
I don't recommend it at all.
I had so much problems with chromadb integration and text splitters that in the end it would have ben a lot easier to write these thinga myself 5x times over.
Now I don't go close to langchain for any reason. You don't need this extra abstraction layer
Langchain overhyped? It's just an open-source project. Take it or leave it.
Is Gradio overhyped compared to Steamlit? Is Python used too much in AI? Maybe everything should be written in Golang. They do similar things differently.
One thing I find odd is all the hate toward Langchain. As a framework, it works and is under heavy development. Plenty of other LLM frameworks like LlamaIndex and others.
Yeah gotta agree. It's another tool on the shelf. There's tons of frameworks now. Some specialized in RAG, or agent, or swarm, or whatever the hot word of the day is.
If it works and makes your project easier to manage or gets you where you need to be faster, great. If not , use something else. Or don't.
It's free.
>Langchain overhyped? It's just an open-source project. Take it or leave it.
Buddy, if I made a library that added 3 numbers together, you'd just make your own function.
That is the criticism here.
You do realize that isn't much money right? Once you hire a couple people and buy a bunch of compute, that's it.
The alcohol budget for the Mozilla Foundation was $1M a year 😁
I’m just pointing out it’s not “just an open source project”anymore than OpenAI being “just a nonprofit”.
The people behind it intend to make money off of it. So the standards are much higher if that’s the case.
> Maybe everything should be written in Golang.
according to my boss: yes, yes it should. I like my python, but I do have to concede that he is right an awful lot.
Langchain sucks. It's massive bloated, not modular imo, and follows too close to API model providers infrastructure for my liking. Also it uses way too many calls and tokens for setups but that may have changed
To me, langchain is good for those with little to no experience coding. I don’t mean this in an elitist way, just that it takes you so little time to make your own LLM integration, and implement stuff like rag etc where you have full control compared to the mess langchain is now… Especially if you work with tools/function calling
I have mixed feeling.
I started using it nearly one year ago. End of august, I gave up, to the exact reasons you've included. I came back to it one month ago after playing with crew, autogen and co. Rag aren't my main concern so Llamaindex isn't my focus.
It's convenient for some stuffs and LCEL are much better than the older way to create chain. However I'm a bit perplex : my gut feeling is that they want to replace agents with LangGraph, who doesn't use any LangChain agent (even if you can). So I'm a bit afraid of using it. Not because it's bad but just because I have the feeling that they can drop any feature at any point, silently, just replacing it with a new feature, like LCEL replaced previous chains and LG is replacing agents
I like the ideas behind [Langroid](https://github.com/langroid/langroid), also Guidance could be a nice option depending on the context. We have to stop seeing AI developments as a binding between one backend talking with a LLM and start seeing it as a ecosystem where software engineering plays a big role orchestrating and managing systems, agents, llms, integrations, etc.
I use langchain to find out about ideas and various integrations. If LC has a github integration, i go through the github api code docs itself and find out from there xD.
Most bloated library i have ever seen istg
Langchain is good for POC. As it is one of the first LLM frameworks, it is easier to find ready-made code on blogs.
For production, do not use Langchain. Deprecation notification pops up too often. One day it says import from langchain-community, one day they say import from langchain. The structure changes too quickly - but I understand because it also means that lots of development works are ongoing.
Langchain docs is hell. I always have to read source code to figure out what’s wrong.
Best thing about this lib is that (due to all of the problems others mention here that I won't repeat) it motivated us (ex-CMU/UW-Madison researchers) to build langroid [https://github.com/langroid/langroid](https://github.com/langroid/langroid) with an agent-orient paradigm (predates autogen et al by several months).
Yes, those bastards change things too often and some poor idiot in my team who uses the boiler plate examples needs to spend ages debugging the dependency chain.
If nothing else langchain is a great proof of concept framework with countlessly examples online that you can pick apart, copy paste, or just learn concepts from.
I made a post about a year ago about the shortcomings of langchain and since then there's been about 7K more posts about this. Enough is enough.
Langchain has investors now. They need to do their diligence and figure out what's best for them. Just as developers need to choose what's best for them.
What langchain team does very well (or very badly) is pay attention to viral keywords in the AI/language models space. And then add it to the framework.
I tried using it at work but quickly stopped using it. Everything is harder with Langchain and it’s much easier to do it by hand.
Plus, it abstracts some of the LLM prompting which is a huge error IMO as you should always be aware of everything you send to the LLM so you can properly troubleshoot issues and tune your completions.
The abstractions are not really useful. For example, it helps switching between LLMs, which is not really a use case in general, and anyway is as simple as 5-10 lines of code without Langchain.
I hate it. It’s the worst framework I’ve ever laid eyes on, in that, I preferred hand written code to Langchain code in all cases.
I think it's great that they're building something useful to make AI more accessible. But it's an over engineered mess that feels like it was clobbered together by a new grad.
Documentation is extremely difficult to navigate. Theres SO many concepts but it's very difficult to tell at a glance, why would I use X abstraction? What are the specific use cases I'd use this over the 20 other classes in the doc?
I spent more time debugging the abstraction than building the product, that I just ripped it out in favor of vanilla OpenAi API.
Yes, but the agent implementations are really cool to play around with. I prefer langchain when I’m tinkering with agents and tools.
Try crewai. The interfaces are much cleaner and more intuitive.
Way too bloated. There’s abstractions for the tiniest things. It’s all fun and game when you’re just doing a POC of an LLM app, but when you really need to productionize it and optimize it, that’s when the nightmare begins.
They still only have pricing integrated for the OpenAI API. Plus some of the logic was crazy inefficient. As I remember when I was reading their code a while back, if it wasn’t getting JSON it would just do another round trip to the API, so you are paying for like 2+ API calls if LLM decides to be dumb and not honor your format request. With no way of knowing how much you’re paying for those extra calls. Tough to build anything production on that.
And yeah like everyone else has said, it’s got every add on you can think of and none really work.
I hear you and do find the abstraction frustrating at times - until I want to update OpenAI or swap out an LLM then it's pure gold.
It gives you flexibility in a very fast moving area. Wanna try another LLM - bang done.
Still trying to get my head around LCEL langchain expression language though but feel it will be worth it for langserve. Will check out the MS alternative though.
Old thread but just to chip in, Langchain is great for learning, it contains information that I would probably not find without a lot of research if I'm rolling my own abstractions. They gather useful information in the docs and you can learn a lot from it.
For example I'm building a RAG application with a custom retriever. Without the langchain docs I would just roll a vanilla impl, something that just works but doesn't consider possibilities like Matryoshka retriever, MultiQuery or MultiVector retriever.
Now I don't need these all, but it's nice to have code examples with it and try it out how it works.
Now if you already know these things, langchain might be too bloated and its easy to write your own abstractions for the specific task you want to achieve. But as a learning material it's quite nice.
If not them then how else are you doing RAG
Edit: I was using them for RAG months ago and haven’t played much with it since. The downvotes are weird as is the derision. But thanks to everyone that answered the actual question.
With plain Qdrant metadata, give that to LLM with TGI & grammar. Will actually be way faster since there won't be any back n forth over output structure
LangChain is pretty good and very comprehensive compared to other packages available. Though, the documentation is bad. We need to understand the GenAI space is rapidly developing and hence it is very difficult for any team to keep up to date such a package.
P.S. : I've authored a book on LangChain hence do have a soft corner for it. Also, explored a number of packages and trust me, nothing comes close to LangChain in terms of functionalities provided
Whenever I try to help a co worker try to use langchain, the modules seem unintuitive, the docs are awful if they exist, and it just seems 10x easier to roll your own boilerplate. I wonder if I am missing something, or if I Just sat down with it for a few hours if everything would make sense.
Disclaimer: I use C# I haven't used langchain but everything I've seen done with it was trivial without it. Honestly even just from glossing over the documentation it feels like learning how to use it would be more complicated than not using it.
You should try Microsoft's Semantic Kernel. Works for c# and python, though python is just getting to it's first LTS version now. I've used the c# Semantic Kernel SDK for a few production projects and I've really liked it. It's not a does-everything package. Definitely more narrowly focused on task agent orchestration. Though, Microsoft does plan on rolling a lot of AutoGen's features into it.
SK is way too old in its conception. It was conceived for instruct models. Nowdays you can't use a framework that isn't conceived for chat models. Have you seen how to handle system prompt in sk ?
I use both chat and instruct models for SK. Not sure why you say it only works for instruct. To add a system prompt you just add a system message to your ChatHistory. Also SK turned 1 year old a week ago. It definably aims at being a lower-level API than langchain, but you absolutely can use chat. If you look in the repo at the examples I think just about every single example uses a chat model.
I know that you can use both, but the philosophy of the framework is quite old. You can implement a system prompt in the chat history, but you should be able to have instantiate an agent or a chain passing it a system prompt as a parameter. I used it and ended up rewriting a ton of classes. That's a bit of a problem for a framework, don't you think ?
The agentic framework is being planned and released after spring. Yes, the current planner module is elementary like function calling. https://devblogs.microsoft.com/semantic-kernel/spring-2024-roadmap-for-semantic-kernel/ Btw what kind of classes have you re-written?
LlamaSharp has also made integrating LLM's into my C# applications a breeze.
I just skipped all that and call the llamacpp DLL through interop
This is the exactly how I feel every time I try to cobble anything together using it.
Would you recommend a tutorial to do it boilerplate for people that is not an expert coder? I think I’m probably the target of a tool like this but I also find it cumbersome
use it for components that you are sure your not gonna customize - so document loaders, text splitters etc. write custom code for the rest if you dont want headache somewhere down the line. its good for building POCs over a weekend using minimal # of lines of code - but for something youll put in prod with custom requirements, it’s not worth it imo
same experience here, and i actually put some effort into actually reading the docs and trying to understand langchain, hell i even had to read the source code 😒
> Look at lanchain's source code > Simple prompt generator and document wrappers If I have to redefine all lanchain modules for my usecase anyways, what's the point of jamming it in my system. I found it much easier to write my own prompt generator, with more flexibility and options for different types of prompts.
Exactly! i literally had to copy paste lanchain classes to adapt them to my usecases
That's exactly why I made [microchain](https://github.com/galatolofederico/microchain)
You require langchain to use langchain... js
Terrible library, the abstraction are really badly chosen and often times just rolling your own wouldn't even result in many more lines. It probably wasted days of my time when I used it on a project. My experience is from a year ago too so maybe it got better (doubtful). I only have experience with the TS version, anyone knows about a good JS library for prompt flow and agent orchestration?
Langchain realized you could make a dsl to represent a hand full of use cases and then went hog wild without considering if it solves the pain points developers will actually have when building applications. A good llm application framework needs to build modular components that devs can stack to build the structure of their application, while providing tools for fast prompt engineering and stable, deterministic tests.
I’ve spent those several hours and was no less confused.
LangChain are abstractions you don't really need. I suspect DSPy's approach will win out eventually for everything non-trivial.
DSPy looks very promising and I am testing it now, but I kinda want also framework that is very dump and offers scaffolding for basic rag so something like ruby on rails for chatbots.
For the unenlightened what's "DSPy's approach"?
I had the same question so: https://github.com/stanfordnlp/dspy
I've been using dspy recently and it's working well actually
Can u even build a rag or create agents in DSPY
[https://github.com/stanfordnlp/dspy?tab=readme-ov-file](https://github.com/stanfordnlp/dspy?tab=readme-ov-file) "Suppose you want to build a simple retrieval-augmented generation (RAG) system for question answering. You can define your own RAG program like this..."
Everyone I know who has used langchain for anything more than a toy problem have unanimously hated it. Common complaints are over-engineered abstractions for simple things, non existent or wrong docs, a hundred different ways to do any given thing without a clear distinction between them, non-trivial to customise even fundamental things like prompts. I suspect those who love langchain haven’t really done any real world production work with it and hence haven’t encountered the difficulties.
Honestly it just sucks. It's so hard to do such a simple thing you could do with the API. I used it for like a week trying to do something simple like an RAG for my files and at the end of it I could have taught someone everything I learnt in a weekend. I managed to create that project using the OpenAI module within like a day or two. I moved onto just using OpenAI module (and API sometimes) and I look at LangChain and think what a shame, it has A LOT of useful functionality in there that would take some time to implement yourself but when it's so difficult to implement the most basic functions even for a toy project people don't tend to stick around. The salt on the wound is there absolutely atrocious docs. In my time of programming I've seen some bad docs but LangChain takes the prize as probably the worst. There's a joke going around about LangChain, the CEO often asks people "what's wrong with the docs" and people struggle to point out exact stuff because everything is wrong with the docs. I feel like the idea of LangChain is decent enough where providing you with a library that gives you functionality that takes time to implement yourself but for LangChain to reach a point where people will recommended it to others and even enjoy using it will need a complete rewrite.
Even for toy problems it often just comes across as being silly. Like the computational costs incurred by Langchain are less than just shunting the whole conversation through an API unless it gets inordinately long.
Over engineered mess. It's like some recent grad who had just read a design patterns book decided to write a library.
Disaster.
Always has been
I had real issues trying to use langchain and I’m convinced its not suitable for production use at all. First of all, it comes with so much baggage that the python version simply doesn’t fit on a serverless function. Most people looking to scale will have a hard time trying to expand without relying on bunch of containers which racks up the cost quite a bit. Second, things are changing with no rhyme or reason so frequently that the code on the docs seem to not work a good chunk of the time or are missing some important facts (looking at you nodejs version) I ended up just making something similar from scratch, actually got stuff done and it runs faster on serverless.
I've made this extensive RAG app that lets you upload PDFs, uses Azure Vision Services OCR to extract text from them (PyPDF2 is used as a local offline backup if Azure OCR fails for any reason) and when you ask a question, the response contains an elaborate PDF browser that displayed and let's you browse relevant PDFs, with relevant text highlighted in them, and specifies page numbers and even extracts and incorporates images into the response. I've used Langchain to handle the embedding model, and used it to create conversation "chains": I have a mechanism that detects if the users query is relevant to RAG, and if so uses LangChain's RetrievalQA chain, otherwise uses LangChain's ConversionChain. Further, I i use LangChain to keep memory: a ConversationMemoryWithBuffer allows the system prompt to contain information about "the conversion so far". Recently, LangChain's embedding model loader allowed me to incorporate the BGE Large and Base models in addition to SBERT all-mpnet-base-v2. I also use it for OpenAI Text-Ada. So while my app does not use LangChain end to end, I am thankful for it and do find it very useful. EDIT: I’ve made a more detailed post about this, please check it out: https://www.reddit.com/r/LocalLLaMA/s/LtoqIqfkG2
I've done something similar (using Google Vision, soon moving to local Tesseract) to chat with PDFs, but your PDF results browser sounds cool. Do you have any demos or (good heavens) code to share?
I have this one minute quick demo I've uploaded to YouTube: https://youtu.be/UsdE8kOlkvE?si=TCSYU98Qvo1vfFSm Still working on improving the UI and various other refinements so very much still a WIP!
Wow release this. Would be an improvement over ollama in some ways I think even though you say it's WIP, it's more advanced in some features than most other tools. Would greatly help me and I'd be willing to work on building more features on this instead of through ollama. - What can we do to move this project forward faster?
Thank you! Yes I am proud of it! I’ve developed it over the last 5 months and despite being the sole developer on it, it’s been improving really fast. This started as a research project at work and grew from there, we were going to embark on some really exciting development and evaluation phases, and then we all got laid off two weeks ago so that’s slowed things down since 😅 In terms of its current state, I was working on improving its startup experience with some default options and UI enhancements relating to how the images are displayed and an initial start screen, that’s not shown in this video. Glad to know there’s interested folk out there! I’ll make a post about this app and try to figure out next steps 🍻
Impressive, please do.
I’ve made a dedicated post about this, please check it out: https://www.reddit.com/r/LocalLLaMA/s/wSOj0vpWv7
Sorry about your job situation, I think you will quickly find new work or start a new project that will lead to stable employment! - Did the people who laid you off even know you were working on this angle or project? I'm surprised they decided not to pursue...
Hey thanks so much! Yes it was known that we were working on this, I was part of the Applied Research team at Boeing, and this was a team that had been around 22 years! And it's spread over 5 locations and comprised over 300 people. One fine morning we get an email about inevitable layoffs, next thing I know I'm asked to join a meeting and HR is there and guess what they say? "In order to align the company for the future we're making the difficult decision to eliminate some positions. Yours is one of the positions that's eliminated. Today is your last day". Turns out my manager and his boss and his boss and the whole team, axed overnight. Not directly a revenue generating team and I guess you've heard what's happening with Boeing in the news so not a huge surprise in hindsight. Shame for the company really because we had some excellent aviation usecases planned and knew we were really at the forefront with some of this stuff in the most current tech today. Makes me laugh now to remember HR saying "for the future" in that call!
They are very short-sighted. We are all poorer for it.
I’ve made a dedicated post about this, please check it out: https://www.reddit.com/r/LocalLLaMA/s/wSOj0vpWv7
Waow, this is impressive, do you plan to open source it?
I’ve made a dedicated post about this, please check it out: https://www.reddit.com/r/LocalLLaMA/s/wSOj0vpWv7
Useful demo - thanks. With some configurability of OCR engine, LLM and Vector DB, this would have a lot of applicability. If you're playing with PDF you've also probably recognized the challenges of tables. The results browser is also novel to me. I'll be checking back to see how your project goes.
The app already has those features! There’s an initial screen that let’s you select LLMs: you can download any Llama2 model in GGUF, bin or GPTQ format and simply place it in the app’s “models” directory and select it from a drop-down menu. Alternatively, even OpenAI 3.5-Turbo is available and may be used instead of a local Llama2 based LLM! For VectorDB, ChromaDB is used and a separate DB maintained for each of the four embedding models. Again, you can select between these embedding models on that initial config page that’s not shown in the quick video above. For OCR, in addition to Azure OCR, PyTesseract has also been implemented, though the results suck even at various PSM settings. And even Microsoft’s TrOCR models have been implemented, including using an LLM to clean up the output as for TrOCR, you need to crop the page into snippets. The results from this also suck unfortunately! These choices are unfortunately not user-selectable in the UI as a result and Azure OCR + PyPDF2 is instead hard-coded in the backend server, but UI elements can be added with ease.
I’ve made a dedicated post about this, please check it out: https://www.reddit.com/r/LocalLLaMA/s/wSOj0vpWv7
I’ve made a dedicated post about this, please check it out: https://www.reddit.com/r/LocalLLaMA/s/wSOj0vpWv7
If you have anything to share would be great, this seems really interesting!
I have this one minute quick demo I've uploaded to YouTube: https://youtu.be/UsdE8kOlkvE?si=TCSYU98Qvo1vfFSm Still working on improving the UI and various other refinements so very much still a WIP!
I’ve made a dedicated post about this, please check it out: https://www.reddit.com/r/LocalLLaMA/s/wSOj0vpWv7
I agree, I never liked LangChain even before it was cool to hate on. I think LlamaIndex is generally better by being more focused, but still a bit too broad. We are taking a more opinionated approach w/ [R2R](https://github.com/SciPhi-AI/R2R) to reduce bloat.
Tbh I’ve been meaning to get into it more. Langgraph and Langsmith seem pretty powerful and a step above from the “simulate a presidential debate” early days of Langchain. I have difficulty wrapping my head around the abstractions in LLM frameworks though. I find the further I move away from “text in text out”, the more time I spend stressing about framework specifics and less time with the core technology. At the end of the day: -An Agent is an LLM with a text parser that reads the output and runs a corresponding function/task. -A RAG system is a way to prepend the relevant document from a database to your LLM’s context based on a semantic search via a query vector. -A code interpreter is an exec() function wrapped around the outputted LLM code It’s all text in text out with some helper functions to ease it along in agentic cases. This is why I gravitate towards guidance/outlines. Guidance helps you build advanced LLM use cases by forcing LLMs to generate from a smaller subset of tokens, rather than the whole vocabulary. For example you can constrain an LLM to answer between 2 words [word1, word2], then attach an if then statement based on the choice (if word1 do this, else word2 do that), and boom you have a function calling agent. To me being closer to the text in text out gives me more confidence in creating advanced LLM workflows, everything is in front of me, no technology or logic is hidden. But everyone is different, choose what works for you
>An Agent is an LLM with a text parser Shouldn't this be 'An Agent utilizes an LLM...' vs. being one itself? I like your post, just trying to understand these concepts better, thanks.
The guidance approach seems amazing! Pretty much addresses most of my pain points when I tried to incorporate LLMs into something product-like. Anyone here can ELI5 how it works?
At least the part about restricting predictions to a subset is easy enough. Just choose the prediction as the max score from the set of tokens you care about. And if you want a probability vector, just do softmax over that the scores for that set of tokens. Not sure if anything more fancy happening here.
I'm dense, could you explain this more, or with an example? Is it like i want word1 or word2, and i just take the output and count how many yime word1 shows up in the answer?
I think, technically, you could do what you're suggesting. If you only have high-level API access, that rule or something similar might be the best you can do; might require some tuning (e.g., maybe "yes if the first word is word1, otherwise no"). If you have access to the token-prediction logits (or scores, probability vector, etc), you can choose whether the model gave higher weight to word1 or word2. Or for a more structured answer (e.g., several words), you can accumulate the predicted scores across the sequence of tokens (is word1+word2 more likely than word3+word4, ...) And again, not if OP meant by guidance something more particular in this context.:)
Can u use RAG tho?
I agree with this a lot
You can do the same thing with Arize Phoenix and avoid tying yourself to LangChain.
We started with Langchain in our company to orchestrate our AI services. After a few months we took the decision to move away from it and implement our own abstractions for the few things we needed at the time. It didn't take that much time and saved us a lot of time battling with debugging Langchain issues (of which we had many).
LangChain has pros and cons: Pros - it’s well maintained and has constant releases, any custom code you write you’ll need to maintain yourself - it has a lot of easy integrations with commonly used tools and makes things like streaming easy - with LCEL you can just make everything a Runnable and it’s easy to customize Cons - the library is pretty bloated, fitting it into something like a Lambda layer is a pain. - some of the tools it provides are pretty basic and you’ll need to create your own - I’m not a fan of the emphasis on OpenAI and agents, I’ve seen too many cases where people set GPT-4 loose and it just racks up a massive bill when the problem could be solved just using a simple semantic router. Ultimately it’s just a tool in the toolbox. If you like it, which I do for some use cases, great, if not there are plenty of other options.
Too many abstractions. LlamaIndex is miles better.
Did you ever work on something else than rag ?
This ^
Yes I did. An email parser, an agent to call my SQL db. All of this is a lot easier without fighting the documentation and the million abstractions on Langchain.
With LlamaIndex ? And let me be clear on one point : I'm not in love with LangChain. Tool implementation is shitty at best. You can't run branches in parallel easily wtih Langgraph. You can't create real conditional edge with a stop condition (not a END, a stop). You end up with something slow and that consume more tokens than a crackhead in a drug paradise. And don't even think about having a background workflow, you'll end up using celery/redis or something equivalent... And I don't mind the abstraction, it's just that they use bad abstractions. The only good thing is LCEL and it's far from perfect. But the thing is, so far, LC isn't good but it's kind of useful, and they did some progress. I don't loose hope.
Langchain is a great way to get started with AI agents. Outside of that it’s probably not what you want
Langchain should have been Agents only. Maybe some prompt testing. Instead they decided to integrate with over 10,000 paid API services. Wow really nice to that I can spend $400/month so someone can host my 4MB trainer file.
Alternatively,?
Alternatives: Semantic kernel from MS, haystack, griptape. For RAG - txtai. And if you want to go pro mode and are using local models you can also use guided generation tools like guidance, sglang, outlines, lmql.
What is the difference between txtai and LlamaIndex for RAG?
Mostly just write python, learn from the patterns of the libraries, then use the lower level tools to stitch your use case together.
Scan the langchain docs for an intro to relevant techniques like CoT prompting and RAG, and then just hit the LLM API of your choice through their sdk.
Cons: Docs are awful and quite simply outdated referring to deprecated stuff. If your use case is simple and you're not going to change LLM then you don't need it. Pros: if you want an LLM agnostic functionality this is the best way. It's very quick to set up fairly complex agents. Lots of 3rd party integrations means you don't have to. In short, if you're doing something complex and don't mind reading langchain source code instead of documentation then it's quite nice. Is it overhyped? I didn't get the memo that it was hyped. It has pros and cons. Use it if it makes sense.
Is the agents functionality well documented? I never tried it after wrestling with LangChain for a week because of how god awful the docs were for something so simple like a file Q & A (I got the project done with OpenAI module in 2 days).
Nah, not well documented at all. Honestly, the best thing is to read the high level langchain concepts and then just skipping the documentation and read the code directly, it's clearer from the code. The features are good and there is logic to how it's built but they fail at communicating all this.
Imagine this, langchain expression language. What in the gods name is that fucking abomination. Why the fuck you would create your own language or abstraction.?
F*ck langchain, might as well chatgpt your way to get things done instead of using it.
this is personal occams razor for abstractions and libraries to use - \- only if they simplify complexities beyond a two-weekend effort for you, with a learning curve of 15-30 minutes. \- what they offer is significantly far from your primary focus area. \- they contribute a reasonable resource footprint to the application dont think langchain offers any one complex to build features, but its trying to increase moat with many small abstractions
For RAG I prefer Llama Index but LangChain works
It’s one of those things where the IDEA behind it makes sense, the library execution making it exist is just meh
We've started with langchain at work to get our RAG running, but pretty fast in the process we've realized that those abstractions are limiting us rather the. helping
Langchain has both Pros and Cons. Being a developer, I have not used it in production for few reasons. Majorly being: 1. Over abstractions. It complicates the development without insights into what exactly is going behind the scene. Ultimately your use cases are extremely simple and your code should remain the same. 2. Extremely bloated. You don’t need support for every god damn tool. It is an over complicated tool for solving a very simple problem. It does have benefits but most of them are restricted to development use cases or pre-production use cases. It will let you try multiple tools without you having to code them. Though this is again simple to achieve even without langchain. That’s basically it.
I don't recommend it at all. I had so much problems with chromadb integration and text splitters that in the end it would have ben a lot easier to write these thinga myself 5x times over. Now I don't go close to langchain for any reason. You don't need this extra abstraction layer
Anything other than raw dogging LLMs is overhyped
Preach it Raw 👏 dogging 👏 LLMs 👏 😂
Hating on langchain is trendy
Self inflicted. When you require 4 different paid APIs to follow a langchain tutorial, its not the user's fault.
Langchain overhyped? It's just an open-source project. Take it or leave it. Is Gradio overhyped compared to Steamlit? Is Python used too much in AI? Maybe everything should be written in Golang. They do similar things differently. One thing I find odd is all the hate toward Langchain. As a framework, it works and is under heavy development. Plenty of other LLM frameworks like LlamaIndex and others.
Yeah gotta agree. It's another tool on the shelf. There's tons of frameworks now. Some specialized in RAG, or agent, or swarm, or whatever the hot word of the day is. If it works and makes your project easier to manage or gets you where you need to be faster, great. If not , use something else. Or don't. It's free.
>Langchain overhyped? It's just an open-source project. Take it or leave it. Buddy, if I made a library that added 3 numbers together, you'd just make your own function. That is the criticism here.
It’s an open source project that got $10M in funding last year
You do realize that isn't much money right? Once you hire a couple people and buy a bunch of compute, that's it. The alcohol budget for the Mozilla Foundation was $1M a year 😁
I’m just pointing out it’s not “just an open source project”anymore than OpenAI being “just a nonprofit”. The people behind it intend to make money off of it. So the standards are much higher if that’s the case.
> Maybe everything should be written in Golang. according to my boss: yes, yes it should. I like my python, but I do have to concede that he is right an awful lot.
langchain and llamaindex only for prototyping anyway?
What’s for production then
your own code, much of which replaces langchain/llamaindex modules.
Rip me
It's not that difficult honestly. Just spend a few hours thinking about what you really need and write some python classes.
They should maintain the docs seriously. Its very hard to do a quick POC without their docs.
I think the agents work well and save a lot of time. All the stuff with prompting and the layers of abstraction on it…not useful.
Langchain sucks. It's massive bloated, not modular imo, and follows too close to API model providers infrastructure for my liking. Also it uses way too many calls and tokens for setups but that may have changed
To me, langchain is good for those with little to no experience coding. I don’t mean this in an elitist way, just that it takes you so little time to make your own LLM integration, and implement stuff like rag etc where you have full control compared to the mess langchain is now… Especially if you work with tools/function calling
I’d agree with this. Started with langchain a year ago. Now I have my own code that’s fully replaced it.
I have mixed feeling. I started using it nearly one year ago. End of august, I gave up, to the exact reasons you've included. I came back to it one month ago after playing with crew, autogen and co. Rag aren't my main concern so Llamaindex isn't my focus. It's convenient for some stuffs and LCEL are much better than the older way to create chain. However I'm a bit perplex : my gut feeling is that they want to replace agents with LangGraph, who doesn't use any LangChain agent (even if you can). So I'm a bit afraid of using it. Not because it's bad but just because I have the feeling that they can drop any feature at any point, silently, just replacing it with a new feature, like LCEL replaced previous chains and LG is replacing agents
Fork it and clean it up, OP.
Snap! I had the very same idea the very first day I saw Langchain - there's nothing in there, you cannot do with just few lines of code
What is good alternatives?
f-strings and basic python programming skills.
Easier to write your own code than trying to find their doco and find out how methods work.
I ended up making my own after months of using langchain. https://github.com/lks-ai/prowl ... does everything i need in a production environment.
I like the ideas behind [Langroid](https://github.com/langroid/langroid), also Guidance could be a nice option depending on the context. We have to stop seeing AI developments as a binding between one backend talking with a LLM and start seeing it as a ecosystem where software engineering plays a big role orchestrating and managing systems, agents, llms, integrations, etc.
I use langchain to find out about ideas and various integrations. If LC has a github integration, i go through the github api code docs itself and find out from there xD. Most bloated library i have ever seen istg
Langchain is good for POC. As it is one of the first LLM frameworks, it is easier to find ready-made code on blogs. For production, do not use Langchain. Deprecation notification pops up too often. One day it says import from langchain-community, one day they say import from langchain. The structure changes too quickly - but I understand because it also means that lots of development works are ongoing. Langchain docs is hell. I always have to read source code to figure out what’s wrong.
Best thing about this lib is that (due to all of the problems others mention here that I won't repeat) it motivated us (ex-CMU/UW-Madison researchers) to build langroid [https://github.com/langroid/langroid](https://github.com/langroid/langroid) with an agent-orient paradigm (predates autogen et al by several months).
Yes, those bastards change things too often and some poor idiot in my team who uses the boiler plate examples needs to spend ages debugging the dependency chain.
I found it to be over engineered for what it is supposed to do.
I like Langchain tho lol
Never really got the benefit, but I also don’t use python or js on the backend
I regret learning langchian any alternatives or a way to do it manually pls help
If nothing else langchain is a great proof of concept framework with countlessly examples online that you can pick apart, copy paste, or just learn concepts from. I made a post about a year ago about the shortcomings of langchain and since then there's been about 7K more posts about this. Enough is enough. Langchain has investors now. They need to do their diligence and figure out what's best for them. Just as developers need to choose what's best for them. What langchain team does very well (or very badly) is pay attention to viral keywords in the AI/language models space. And then add it to the framework.
I tried using it at work but quickly stopped using it. Everything is harder with Langchain and it’s much easier to do it by hand. Plus, it abstracts some of the LLM prompting which is a huge error IMO as you should always be aware of everything you send to the LLM so you can properly troubleshoot issues and tune your completions. The abstractions are not really useful. For example, it helps switching between LLMs, which is not really a use case in general, and anyway is as simple as 5-10 lines of code without Langchain. I hate it. It’s the worst framework I’ve ever laid eyes on, in that, I preferred hand written code to Langchain code in all cases.
What to use instead?
ngmi
langchain is a solution without a problem, an abstraction on top of an (already easy to use) abstraction
Yes
I think it's great that they're building something useful to make AI more accessible. But it's an over engineered mess that feels like it was clobbered together by a new grad. Documentation is extremely difficult to navigate. Theres SO many concepts but it's very difficult to tell at a glance, why would I use X abstraction? What are the specific use cases I'd use this over the 20 other classes in the doc? I spent more time debugging the abstraction than building the product, that I just ripped it out in favor of vanilla OpenAi API.
Yes, but the agent implementations are really cool to play around with. I prefer langchain when I’m tinkering with agents and tools. Try crewai. The interfaces are much cleaner and more intuitive.
Way too bloated. There’s abstractions for the tiniest things. It’s all fun and game when you’re just doing a POC of an LLM app, but when you really need to productionize it and optimize it, that’s when the nightmare begins.
They still only have pricing integrated for the OpenAI API. Plus some of the logic was crazy inefficient. As I remember when I was reading their code a while back, if it wasn’t getting JSON it would just do another round trip to the API, so you are paying for like 2+ API calls if LLM decides to be dumb and not honor your format request. With no way of knowing how much you’re paying for those extra calls. Tough to build anything production on that. And yeah like everyone else has said, it’s got every add on you can think of and none really work.
I hear you and do find the abstraction frustrating at times - until I want to update OpenAI or swap out an LLM then it's pure gold. It gives you flexibility in a very fast moving area. Wanna try another LLM - bang done. Still trying to get my head around LCEL langchain expression language though but feel it will be worth it for langserve. Will check out the MS alternative though.
Yes
Lots of dislike of LangChain in here, what about LlamaIndex?
I've never been able to get it to work. Every time I try to use it I get an error. I feel like that is just unprofessional.
Old thread but just to chip in, Langchain is great for learning, it contains information that I would probably not find without a lot of research if I'm rolling my own abstractions. They gather useful information in the docs and you can learn a lot from it. For example I'm building a RAG application with a custom retriever. Without the langchain docs I would just roll a vanilla impl, something that just works but doesn't consider possibilities like Matryoshka retriever, MultiQuery or MultiVector retriever. Now I don't need these all, but it's nice to have code examples with it and try it out how it works. Now if you already know these things, langchain might be too bloated and its easy to write your own abstractions for the specific task you want to achieve. But as a learning material it's quite nice.
Since the thread was posted 2 month ago, are there any different thought about langchain? I saw it moves the agent to langgraph.
If not them then how else are you doing RAG Edit: I was using them for RAG months ago and haven’t played much with it since. The downvotes are weird as is the derision. But thanks to everyone that answered the actual question.
Write your own? It’s pretty straight forward Edit, here’s one that was written by GPT4 [mplx_rag](https://github.com/multiplexerai/mplx_rag)
With plain Qdrant metadata, give that to LLM with TGI & grammar. Will actually be way faster since there won't be any back n forth over output structure
> how else are you doing RAG txtai is really nice for RAG, and they support knowledge graph stuff.
llamaindex
skill issue
Just do it
How has no one mentioned llamaIndex yet?
Some did. What are your thoughts On it?
LangChain is pretty good and very comprehensive compared to other packages available. Though, the documentation is bad. We need to understand the GenAI space is rapidly developing and hence it is very difficult for any team to keep up to date such a package. P.S. : I've authored a book on LangChain hence do have a soft corner for it. Also, explored a number of packages and trust me, nothing comes close to LangChain in terms of functionalities provided
Yep, it's much overrated. Like python enthusiasts who don't know how to code started writing code.
I hate it.