LangChain4j实战

基础

LangChain4j模型适配:

Provider	Native Image	Sync Completion	Streaming Completion	Embedding	Image Generation	Scoring	Function Calling
OpenAI	✅	✅	✅	✅	✅		✅
Azure OpenAI		✅	✅	✅	✅		✅
Hugging Face		✅		✅
Amazon Bedrock		✅	✅	✅	✅
Google Vertex AI Gemini		✅	✅		✅		✅
Google Vertex AI	✅	✅		✅	✅
Mistral AI		✅	✅	✅			✅
DashScope		✅	✅	✅
LocalAI		✅	✅	✅			✅
Ollama		✅	✅	✅
Cohere						✅
Qianfan		✅	✅	✅			✅
ChatGLM		✅
Nomic				✅
Anthropic	✅	✅	✅				✅
Zhipu AI		✅	✅	✅			✅

Langchain实战应用场景:

https://github.com/lyhue1991/eat_chatgpt/blob/main/3_langchain_9_usecases.ipynb

官方文档:

LangChain4j | LangChain4j

官方LangChain4j使用示例

https://github.com/langchain4j/langchain4j-examples.git

基本使用示例是比较完整

官方SpringBoot整合示例

Spring Boot Integration | LangChain4j

SpringBoot实战使用示例

自己写的demo, 使用LangChain4j为内部系统生成有意义的测试数据

https://github.com/kvenLin/spring-langchain-demo

LangChain4j的实战用例讲解

核心功能

LangChain4j已经将很多和大模型进行复杂的操作进行了简化,对于后端程序员只需要调用指定的API即可

个人觉得还是需要提前理解LangChain的基础架构, 了解每个模块的作用, 使用起来才会更容易

LangChain4j主要的核心的几个功能就是:

Chat and Language Models: 切换大模型
Chat Memory: 对系统内聊天指定聊天memoryId进行区分, 可以根据memoryId来持续对话内容
Model Parameters: 根据您选择的模型型号和提供程序,可以调整许多参数
Response Streaming: 响应流式处理, LLM提供程序提供了一种逐个令牌流式传输响应的方法，而不是等待生成整个文本
AI Services: 比较高级的核心功能, 通过代理的形式帮我们实现特定的Service层的服务, 只需要关注业务, 不需要关注底层实现
Tools (Function Calling): 除了生成文本外，还可以触发操作。在tools层可以根据触发条件调用不同的函数, 也是比较核心的模块, 对于AI-Agent的实现是必须的.
RAG (Retrieval-Augmented Generation): 根据特定的文档+向量化数据, 来扩展模型的知识库, 提高搜索的有效性
Structured Data Extraction: 这里官方文档还没有完善, 但是examples中可以找打使用示例, 主要的作用是根据文本数据抽取我们需要的信息, 并封装成对Java有意义的自定义的结构化对象.
Classification: 官方文档待完善...
Embedding (Vector) Stores: 向量数据的存储功能, 提供了很多示例, 可以使用ES、Redis
Chains: 官方文档待完善...
Image Models: 官方文档待完善...

使用教程

AI Service

工作原理

官方解释: 将接口与低级组件一起提供 Class ， AiServices 并 AiServices 创建实现此接口的代理对象。目前，它使用反射，但我们也在考虑替代方案。此代理对象处理输入和输出的所有转换。在本例中，输入是单个 String ，但我们使用ChatLanguageModel 作为 ChatMessage 输入。因此， AiService 会自动将其转换为 UserMessage 并调用 ChatLanguageModel .由于 chat 该方法的输出类型是 String ，在返回 AiMessage 后 ChatLanguageModel ，它会在从 chat 方法返回之前转换为 String。

实际上就是代理形式帮我们实现了定义的业务接口

POJO抽取

public class POJO_Extracting_AI_Service_Example {
    static ChatLanguageModel model = OpenAiChatModel.builder()
            .baseUrl(ApiKeys.BASE_URL)
            .apiKey(ApiKeys.OPENAI_API_KEY)
            .logRequests(true)
            .logResponses(true)
            .timeout(ofSeconds(60))
            .build();

    static class Person {

        private String firstName;
        private String lastName;
        private LocalDate birthDate;

        @Override
        public String toString() {
            return "Person {" +
                    " firstName = \"" + firstName + "\"" +
                    ", lastName = \"" + lastName + "\"" +
                    ", birthDate = " + birthDate +
                    " }";
        }
    }

    interface PersonExtractor {

        @UserMessage("Extract information about a person from {{it}}")
        Person extractPersonFrom(String text);
    }

    public static void main(String[] args) {

        PersonExtractor extractor = AiServices.create(PersonExtractor.class, model);

        String text = "In 1968, amidst the fading echoes of Independence Day, "
                + "a child named John arrived under the calm evening sky. "
                + "This newborn, bearing the surname Doe, marked the start of a new journey.";

        Person person = extractor.extractPersonFrom(text);

        System.out.println(person); // Person { firstName = "John", lastName = "Doe", birthDate = 1968-07-04 }
    }
}

自定义的业务需要的对象: Person对象
定义业务接口: PersonExtractor
@UserMessage标注当前接口方法使用来做什么的
text 会自动预处理, 在 @UserMessage 中进行替换{{it}}

最后得到的就是Service自动从文本中抽取数据并自动构建的Person对象

应用场景:

可以对用户模糊描述提取有用的信息, 进行精确的业务处理
对文档提取特定的数据进行业务处理

@SystemMessage的使用

限定AI角色区分service不同函数实现功能

public class AI_Service_with_System_and_User_Messages_Example {
    static ChatLanguageModel model = OpenAiChatModel.builder()
            .baseUrl(ApiKeys.BASE_URL)
            .apiKey(ApiKeys.OPENAI_API_KEY)
            .logRequests(true)
            .logResponses(true)
            .timeout(ofSeconds(60))
            .build();
    
    interface TextUtils {

        @SystemMessage("You are a professional translator into {{language}}")
        @UserMessage("Translate the following text: {{text}}")
        String translate(@V("text") String text, @V("language") String language);

        @SystemMessage("Summarize every message from user in {{n}} bullet points. Provide only bullet points.")
        List<String> summarize(@UserMessage String text, @V("n") int n);
    }

    public static void main(String[] args) {

        TextUtils utils = AiServices.create(TextUtils.class, model);

        String translation = utils.translate("Hello, how are you?", "italian");
        System.out.println(translation); // Ciao, come stai?

        String text = "AI, or artificial intelligence, is a branch of computer science that aims to create "
                + "machines that mimic human intelligence. This can range from simple tasks such as recognizing "
                + "patterns or speech to more complex tasks like making decisions or predictions.";

        List<String> bulletPoints = utils.summarize(text, 3);
        bulletPoints.forEach(System.out::println);
        // [
        // "- AI is a branch of computer science",
        // "- It aims to create machines that mimic human intelligence",
        // "- It can perform simple or complex tasks"
        // ]
    }
}

文本分析情感

根据文本内容分析情感色彩, 积极、中立、消极

public class Sentiment_Extracting_AI_Service_Example {
    static ChatLanguageModel model = OpenAiChatModel.builder()
            .baseUrl(ApiKeys.BASE_URL)
            .apiKey(ApiKeys.OPENAI_API_KEY)
            .logRequests(true)
            .logResponses(true)
            .timeout(ofSeconds(60))
            .build();

    enum Sentiment {
        POSITIVE, NEUTRAL, NEGATIVE;
    }

    interface SentimentAnalyzer {

        @UserMessage("Analyze sentiment of {{it}}")
        Sentiment analyzeSentimentOf(String text);

        @UserMessage("Does {{it}} have a positive sentiment?")
        boolean isPositive(String text);
    }

    public static void main(String[] args) {

        SentimentAnalyzer sentimentAnalyzer = AiServices.create(SentimentAnalyzer.class, model);

        Sentiment sentiment = sentimentAnalyzer.analyzeSentimentOf("It is good!");
        System.out.println(sentiment); // POSITIVE

        boolean positive = sentimentAnalyzer.isPositive("It is bad!");
        System.out.println(positive); // false
    }
}

文本数据类型转换

public class Number_Extracting_AI_Service_Example {
    static ChatLanguageModel model = OpenAiChatModel.builder()
            .baseUrl(ApiKeys.BASE_URL)
            .apiKey(ApiKeys.OPENAI_API_KEY)
            .logRequests(true)
            .logResponses(true)
            .timeout(ofSeconds(60))
            .build();

    interface NumberExtractor {

        @UserMessage("Extract number from {{it}}")
        int extractInt(String text);

        @UserMessage("Extract number from {{it}}")
        long extractLong(String text);

        @UserMessage("Extract number from {{it}}")
        BigInteger extractBigInteger(String text);

        @UserMessage("Extract number from {{it}}")
        float extractFloat(String text);

        @UserMessage("Extract number from {{it}}")
        double extractDouble(String text);

        @UserMessage("Extract number from {{it}}")
        BigDecimal extractBigDecimal(String text);
    }

    public static void main(String[] args) {

        NumberExtractor extractor = AiServices.create(NumberExtractor.class, model);

        String text = "After countless millennia of computation, the supercomputer Deep Thought finally announced "
                + "that the answer to the ultimate question of life, the universe, and everything was forty two.";

        int intNumber = extractor.extractInt(text);
        System.out.println(intNumber); // 42

        long longNumber = extractor.extractLong(text);
        System.out.println(longNumber); // 42

        BigInteger bigIntegerNumber = extractor.extractBigInteger(text);
        System.out.println(bigIntegerNumber); // 42

        float floatNumber = extractor.extractFloat(text);
        System.out.println(floatNumber); // 42.0

        double doubleNumber = extractor.extractDouble(text);
        System.out.println(doubleNumber); // 42.0

        BigDecimal bigDecimalNumber = extractor.extractBigDecimal(text);
        System.out.println(bigDecimalNumber); // 42.0
    }
}

public class Date_and_Time_Extracting_AI_Service_Example {
    static ChatLanguageModel model = OpenAiChatModel.builder()
            .baseUrl(ApiKeys.BASE_URL)
            .apiKey(ApiKeys.OPENAI_API_KEY)
            .logRequests(true)
            .logResponses(true)
            .timeout(ofSeconds(60))
            .build();

    interface DateTimeExtractor {

        @UserMessage("Extract date from {{it}}")
        LocalDate extractDateFrom(String text);

        @UserMessage("Extract time from {{it}}")
        LocalTime extractTimeFrom(String text);

        @UserMessage("Extract date and time from {{it}}")
        LocalDateTime extractDateTimeFrom(String text);
    }

    public static void main(String[] args) {

        DateTimeExtractor extractor = AiServices.create(DateTimeExtractor.class, model);

        String text = "The tranquility pervaded the evening of 1968, just fifteen minutes shy of midnight,"
                + " following the celebrations of Independence Day.";

        LocalDate date = extractor.extractDateFrom(text);
        System.out.println(date); // 1968-07-04

        LocalTime time = extractor.extractTimeFrom(text);
        System.out.println(time); // 23:45

        LocalDateTime dateTime = extractor.extractDateTimeFrom(text);
        System.out.println(dateTime); // 1968-07-04T23:45
    }
}

区分对话和记忆

public class ServiceWithMemoryExample {
    static ChatLanguageModel model = OpenAiChatModel.builder()
            .baseUrl(ApiKeys.BASE_URL)
            .apiKey(ApiKeys.OPENAI_API_KEY)
            .logRequests(true)
            .logResponses(true)
            .timeout(ofSeconds(60))
            .build();

    interface Assistant {
        String chat(@MemoryId int memoryId, @UserMessage String userMessage);
    }

    public static void main(String[] args) {

        Assistant assistant = AiServices.builder(Assistant.class)
                .chatLanguageModel(model)
                .chatMemoryProvider(memoryId -> MessageWindowChatMemory.withMaxMessages(10))
                .build();

        System.out.println(assistant.chat(1, "Hello, my name is Klaus"));
        // Hi Klaus! How can I assist you today?

        System.out.println(assistant.chat(2, "Hello, my name is Francine"));
        // Hello Francine! How can I assist you today?

        System.out.println(assistant.chat(1, "What is my name?"));
        // Your name is Klaus.

        System.out.println(assistant.chat(2, "What is my name?"));
        // Your name is Francine.
    }
}

AI Tools

简单使用

public class _10_ServiceWithToolsExample {

    // Please also check CustomerSupportApplication and CustomerSupportApplicationTest
    // from spring-boot-example module

    static class Calculator {

        @Tool("Calculates the length of a string")
        int stringLength(String s) {
            System.out.println("Called stringLength() with s='" + s + "'");
            return s.length();
        }

        @Tool("Calculates the sum of two numbers")
        int add(int a, int b) {
            System.out.println("Called add() with a=" + a + ", b=" + b);
            return a + b;
        }

        @Tool("Calculates the square root of a number")
        double sqrt(int x) {
            System.out.println("Called sqrt() with x=" + x);
            return Math.sqrt(x);
        }
    }

    interface Assistant {

        String chat(String userMessage);
    }

    public static void main(String[] args) {

        ChatLanguageModel model = OpenAiChatModel.builder()
                .baseUrl(ApiKeys.BASE_URL)
                .apiKey(ApiKeys.OPENAI_API_KEY)
                .logRequests(false)
                .build();

        Assistant assistant = AiServices.builder(Assistant.class)
                .chatLanguageModel(model)
                .tools(new Calculator())
                .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
                .build();

        String question = "What is the square root of the sum of the numbers of letters in the words \"hello\" and \"world\"?";

        String answer = assistant.chat(question);

        System.out.println(answer);
        // The square root of the sum of the number of letters in the words "hello" and "world" is approximately 3.162.
    }
}

@Tool: 添加对工具的描述, 告诉AI这个方法是的作用是什么
AiServices构建的时候添加tools类, 模型就知道这个工具什么时候调用
当chat输入文本内容和tools中工具的方法含义相同时, 就会调用自定义的工具方法的函数进行处理得到结果

因为有些模型计算逻辑的处理并不是很好, 这样使用自定义的工具可以进行复杂的逻辑处理, AI只需要根据工具调用不同的方法拿到结果告诉用户即可

SpringBoot中进行使用

参考demo

GitHub - kvenLin/spring-langchain-demo: use LangChain4j dynamic generate meaningful test data for database

spring.application.name=spring-langchain-demo
langchain4j.open-ai.chat-model.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.chat-model.base-url=${OPENAI_API_URL}
langchain4j.open-ai.chat-model.model-name=gpt-3.5-turbo
langchain4j.open-ai.chat-model.temperature=0.7
# 开启调用open-ai请求日志
langchain4j.open-ai.chat-model.log-requests=true
# 开启调用open-ai响应日志
langchain4j.open-ai.chat-model.log-responses=true
logging.level.dev.langchain4j=DEBUG
logging.level.dev.ai4j.openai4j=DEBUG
server.port=8081
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/spring-langchain-demo?useSSL=false&useUnicode=true&characterEncoding=UTF-8
spring.datasource.username=root
spring.datasource.password=123456
# MyBatis-Plus configuration
mybatis-plus.configuration.map-underscore-to-camel-case=true
mybatis-plus.configuration.auto-mapping-behavior=full
mybatis-plus.configuration.log-impl=org.apache.ibatis.logging.stdout.StdOutImpl
mybatis-plus.mapper-locations=classpath*:mapper/**/*Mapper.xml
mybatis-plus.global-config.db-config.logic-not-delete-value=1
mybatis-plus.global-config.db-config.logic-delete-value=0

这里的 ${OPENAI_API_KEY} 和 ${OPENAI_API_URL} 可以用系统环境变量的方式提供

国内使用open-ai的模型有一定限制, 所以这里使用的是三方代理地址: F2API - OpenAI API Key

自定义配置:

package com.louye.springlangchaindemo.config;

import com.louye.springlangchaindemo.service.ai.AssistantService;
import com.louye.springlangchaindemo.service.ai.Factory;
import com.louye.springlangchaindemo.tool.AssistantTools;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.openai.OpenAiChatModel;
import dev.langchain4j.service.AiServices;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import static java.time.Duration.ofSeconds;

@Configuration
class AssistantConfiguration {

    @Bean
    AssistantService assistantService(ChatLanguageModel chatLanguageModel, AssistantTools assistantTools) {
        return AiServices.builder(AssistantService.class)
                .chatLanguageModel(chatLanguageModel)
                .chatMemory(MessageWindowChatMemory.builder()
                        .maxMessages(10).build())
                .tools(assistantTools)
                .build();
    }

    @Bean
    Factory factoryService() {
        ChatLanguageModel chatLanguageModel = OpenAiChatModel.builder()
                .baseUrl(System.getenv("OPENAI_API_URL"))
                .apiKey(System.getenv("OPENAI_API_KEY"))
                .timeout(ofSeconds(60))
//                .responseFormat("json_object")
                .build();
        return AiServices.create(Factory.class, chatLanguageModel);
    }



}

ai-service定义:

package com.louye.springlangchaindemo.service.ai;

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.spring.AiService;

@AiService
public interface AssistantService {

    @SystemMessage("""
        you are system assistant, you can help me to do some works in this system.
        if user want to generate table data, must input the table name and the number of rows.
        """)
    String chat(String message);
}

public interface Factory {

    ProductDataList generateTestDataForProduct(TableDataGeneratePrompt prompt);

    CartDataList generateTestDataForCart(TableDataGeneratePrompt prompt);

    UserDataList generateTestDataForUser(TableDataGeneratePrompt prompt);
}

tools自定义: 让用户提供表名和新增数据量, tools会根据用户指定的表去对该表新增测试数据

package com.louye.springlangchaindemo.tool;

import cn.hutool.core.collection.CollUtil;
import cn.hutool.core.util.StrUtil;
import com.louye.springlangchaindemo.domain.Product;
import com.louye.springlangchaindemo.domain.aidata.CartDataList;
import com.louye.springlangchaindemo.domain.aidata.ProductDataList;
import com.louye.springlangchaindemo.domain.aidata.UserDataList;
import com.louye.springlangchaindemo.service.CartService;
import com.louye.springlangchaindemo.service.ProductService;
import com.louye.springlangchaindemo.service.UserService;
import com.louye.springlangchaindemo.service.ai.Factory;
import com.louye.springlangchaindemo.template.TableDataGeneratePrompt;
import dev.langchain4j.agent.tool.P;
import dev.langchain4j.agent.tool.Tool;
import jakarta.annotation.Resource;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;

import java.time.LocalTime;
import java.util.List;

@Slf4j
@Component
public class AssistantTools {
    @Resource
    private Factory factory;
    @Resource
    private ProductService productService;
    @Resource
    private CartService cartService;
    @Resource
    private UserService userService;

    @Tool
    public String currentTime() {
        return LocalTime.now().toString();
    }

    @Tool("when user need to open the system")
    public String openSystem() {
        log.info("user need to open the system, do something here");
        return "success";
    }

    @Tool("when user need to generate test data for aim table")
    public String generateTestData(@P("tableName to generate test data") String tableName,
                                   @P("number of rows to generate") Integer num) {
        log.info("query table structure");
        String createTableDDL = userService.showTableDDL(tableName);
        if (StrUtil.isEmpty(createTableDDL)) {
            throw new RuntimeException("table not exisdt");
        }
        log.info("query table max id");
        Integer maxId = userService.maxIdForTable(tableName);
        TableDataGeneratePrompt prompt = new TableDataGeneratePrompt(tableName, num, maxId);
        if (tableName.equals("user")) {
            UserDataList userDataList = factory.generateTestDataForUser(prompt);
            log.info("userDataList: {}", userDataList);
            if (CollUtil.isNotEmpty(userDataList.getUserList())) {
                userService.saveBatch(userDataList.getUserList());
            }
            return userDataList.toString();
        } else if (tableName.equals("product")) {
            ProductDataList productDataList = factory.generateTestDataForProduct(prompt);
            log.info("productDataList: {}", productDataList);
            if (CollUtil.isNotEmpty(productDataList.getProductList())) {
                productService.saveBatch(productDataList.getProductList());
            }
            return productDataList.toString();
        }else if (tableName.equals("cart")) {
            CartDataList cartDataList = factory.generateTestDataForCart(prompt);
            log.info("cartDataList: {}", cartDataList);
            if (CollUtil.isNotEmpty(cartDataList.getCartList())) {
                cartService.saveBatch(cartDataList.getCartList());
            }
            return cartDataList.toString();
        }
        return "no handle tool for this table:" + tableName;
    }

}

controller实现:

package com.louye.springlangchaindemo.controller;

import com.louye.springlangchaindemo.tool.AssistantTools;
import com.louye.springlangchaindemo.service.ai.AssistantService;
import jakarta.annotation.Resource;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;


@Slf4j
@RequestMapping("/assistant")
@RestController
class AssistantController {

    @Resource
    private AssistantService assistant;

    @Resource
    private AssistantTools assistantTools;


    @GetMapping("/chat")
    public String chat(@RequestParam(value = "message", defaultValue = "What is the time now?") String message) {
        log.info("AssistantController.chat() called with message: {}", message);
        return assistant.chat(message);
    }
}

测试:

个人想法

后续AI的发展必然是AI-Agent的方向, 但是目前agent的工具大多都是提升生产力的工具, 而偏向系统使用方向的agent不太多, 感觉可能需要让AI明白一个系统如何使用并没有那么容易, 只有系统内部改造才能让AI明白什么时候调用什么函数, 这或许又涉及到业务重构等问题.

大方向上, 后续可能就不太需要用户再去在网站上点击操作了, 而是对话形式进行自己的业务处理改变数据库数据.但是这可能就涉及一些AI安全方向的, 可能这也是为什么目前为止还没有一些成熟的系统网站的agent的实现.
LangChain4j对于Java程序员来说, 更多的是提供了一种新的思路去设计系统的业务处理模式.期待后续LangChain4j的完善.