SpringCloudAliaba生成式JavaAI应用开发文生问答音像

采用SpringCloudAliabaAI型式大模型LLM，进行生成式JavaAI应用开发，实现文生问答、图像和语音合成，Web应用页面交互展现。SpringBootGradle软件框架，Idea集成开发环境，API_Post嵌入插件一体测试。

1 工效展示[文生-答/图/音]

2 软件体系架构[SpringBootGradle]

3 Idea开发环境项目导入

4 SpringAlibabaAI依赖添加

5 application.yml调度配置

6 AI服务支撑

6.1 AI答/图/音服务接口编码

6.2 AI答/图/音服务实现编码

关键的LLM运用编码如下：

@Override

public String completion(String message) {

Prompt prompt = new Prompt(new UserMessage(message));

return chatClient.call(prompt).getResult().getOutput().getContent();

}

@Override

public ImageResponse genImg(String imgPrompt) {

var prompt = new ImagePrompt(imgPrompt);

return imageClient.call(prompt);

}

@Override

public Map<String, String> streamCompletion(String message) {

StringBuilder fullContent = new StringBuilder();

streamingChatClient.stream(new Prompt(message))

.flatMap(chatResponse -> Flux.fromIterable(chatResponse.getResults()))

.map(content -> content.getOutput().getContent())

.doOnNext(fullContent::append)

.last()

.map(lastContent -> Map.of(message, fullContent.toString()))

.block();

return Map.of(message, fullContent.toString());

}

@Override

public String genAudio(String text) {

var resWAV = speechClient.call(text);

return save(resWAV, SpeechSynthesisAudioFormat.WAV.getValue());

}

private String save(ByteBuffer audio, String type) { // 将语音保存到本地的方法

String currentPath = System.getProperty("user.dir");

LocalDateTime now = LocalDateTime.now();

DateTimeFormatter formatter = DateTimeFormatter.ofPattern("MM-dd-HH-mm-ss");

String fileName = currentPath + File.separator + now.format(formatter) + "." + type;

File file = new File(fileName);

try (FileOutputStream fos = new FileOutputStream(file)) {

fos.write(audio.array());

} catch (Exception e) {

throw new RuntimeException(e);

}

return fileName;

}

6.3 Saas服务/Web展现编码

核心的文生问答/音像部分的SaaS支撑编码如下：

private final TongYiService tongYiService;

public TongYiController(TongYiService tongYiService) {

this.tongYiService = tongYiService;

}

@GetMapping("/chat")

public String completion(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {

return tongYiService.completion(message);

}

@GetMapping("/stream")

public Map<String, String> streamCompletion(@RequestParam(

value = "message", defaultValue = "请告诉我西红柿炖牛腩怎么做？") String message) {

return tongYiService.streamCompletion(message);

}

@GetMapping("/genImg")

public ImageResponse genImg(@RequestParam(value = "prompt",

defaultValue = "Painting a picture of blue water and blue sky.") String imgPrompt) {

return tongYiService.genImg(imgPrompt);

}

@GetMapping("/getImgUrl")

public String getImgUrl(@RequestParam(value = "prompt",

defaultValue = "Painting a picture of blue water and blue sky.") String imgPrompt) {

ImageResponse imageResponse = tongYiService.genImg(imgPrompt);

return imageResponse.getResult().getOutput().getUrl();

}

@GetMapping("/getImgEtt")

public ResponseEntity<byte[]> getImgEtt(@RequestParam(value = "msg",

defaultValue = "动漫女主图片") String imgPrompt){

ImageResponse imageResponse = tongYiService.genImg(imgPrompt);

String b64Json = imageResponse.getResult().getOutput().getB64Json();

byte[] decode = Base64.getDecoder().decode(b64Json);

HttpHeaders httpHeaders = new HttpHeaders();

httpHeaders.setContentType(MediaType.IMAGE_PNG);

return new ResponseEntity<>(decode,httpHeaders, HttpStatus.OK);

}

@GetMapping("/genAudio")

public String genAudio(@RequestParam(value = "prompt",

defaultValue = "你好，Spring Cloud Alibaba AI 框架！") String prompt) {

return tongYiService.genAudio(prompt);

}

@GetMapping("/speech")

public void speech(@RequestParam(value = "prompt", defaultValue = "Tell me a joke") String prompt,

HttpServletResponse response) throws IOException {

String audio = tongYiService.genAudio(prompt);

FileInputStream is = new FileInputStream(audio);

int i = is.available(); // 得到文件大小

byte[] data = new byte[i];

is.read(data); // 读数据

is.close();

response.setContentType("audio/wav"); // 设置返回的文件类型

OutputStream toClient = response.getOutputStream();

toClient.write(data); // 向客户端输出二进制数据的对象数据

toClient.close();

}

//文字输入语音输出答案

@GetMapping("/cvtMp3")

public void cvtMp3(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message,

HttpServletResponse response) throws IOException {

String completion = tongYiService.completion(message);

String audio = tongYiService.genAudio(completion);

FileInputStream is = new FileInputStream(audio);

int i = is.available(); // 得到文件大小

byte[] data = new byte[i];

is.read(data); // 读数据

is.close();

response.setContentType("audio/wav"); // 设置返回的文件类型

OutputStream toClient = response.getOutputStream();

toClient.write(data); // 向客户端输出二进制数据的对象数据

toClient.close();

}

7 服务功能测调试

逐一测试，核心的文生问答/音像部分的控制器SaaS支撑服务调用函数，下面列出不同方式的典型截图。

7.1 API_Post

Idea插件API_Post，针对getImg的测试截图，如下：

完全的API_Post环境，针对getImg的测试截图，如下：

7.2 浏览器端

8 Web交互应用实现

8.1 文生问答

单轮、多轮或流式文生问答展现，以多轮文生问答为例，相关编码如下：

<!DOCTYPE html>

<head>

maximum-scale=1, minimum-scale=1, width=device-width">

<style>

body { background-color: #f8f9fa; font-family: Arial, sans-serif; }

.container { margin: 50px auto; width: 800px; background-color: #fff;

padding: 20px; border-radius: 5px; box-shadow: 0 2px 5px rgba(0, 0, 0, 0.1) }

h1 { color: #2ecc71; text-align: center; margin-bottom: 30px; }

label { display: block; margin-bottom: 10px; color: #333 }

input[type="text"] { width: 100%; padding: 10px; border: 1px solid #ccc; border-radius: 3px }

input[type="submit"] { background-color: #2ecc71; color: #fff;

border: none; padding: 10px 20px; border-radius: 3px; cursor: pointer; width: 100% }

.chat-box { width: 100%; height: 500px; padding: 10px;

border: 1px solid #ccc; border-radius: 3px; overflow-y: scroll }

.loader { text-align: center }

.loader::after { content: ""; display: inline-block; width: 20px; height: 20px; border-radius: 50%;

border: 2px solid #ccc; border-top-color: #2ecc71; animation: spin 1s infinite linear }

@keyframes spin { to { transform: rotate(360deg) } }

</style>

</head>

<body>

<h1>Spring Cloud Alibaba AI</h1>

</form>

<br>

</div>

</body>

let loader = document.getElementById("loader")

document.getElementById("form").addEventListener("submit", function(event) {

event.preventDefault()

let messageInput = document.getElementById("message")

let message = messageInput.value; messageInput.value = ""

let chatBox = document.getElementById("chat-box")

let userMessage = document.createElement("div")

userMessage.className = "message user-message"

userMessage.textContent = "用户: " + message

chatBox.appendChild(userMessage)

chatBox.scrollTop = chatBox.scrollHeight

loader.style.display = "block"

let xhr = new XMLHttpRequest()

xhr.open("GET", "http://localhost:9320/chat?message=" + encodeURIComponent(message), true)

xhr.onreadystatechange = function() {

if (xhr.readyState === 4) {

loader.style.display = "none"

if (xhr.status === 200) {

let response = xhr.responseText, botMessage = document.createElement("div")

botMessage.className = "message bot-message"

let botMessageText = document.createElement("span")

botMessageText.className = "message-text"; botMessage.appendChild(botMessageText)

botMessageText.innerHTML = marked.marked(response)

chatBox.appendChild(botMessage); chatBox.scrollTop = chatBox.scrollHeight

} else if (xhr.status === 400) {

let error = JSON.parse(xhr.responseText)

let errorMessage = document.createElement("div")

errorMessage.className = "message bot-message"

errorMessage.textContent = "Bot: " + error.message

chatBox.appendChild(errorMessage); chatBox.scrollTop = chatBox.scrollHeight

} else {

let errorMessage = document.createElement("div");

errorMessage.className = "message bot-message";

errorMessage.textContent = "Bot: Failed to connect to the backend service. "

+ "Please make sure the backend service is running.";

chatBox.appendChild(errorMessage); chatBox.scrollTop = chatBox.scrollHeight;

}

xhr.onloadstart = function() { loader.style.display = "block" }

xhr.onloadend = function() { loader.style.display = "none" }; xhr.send()

})

</script>

</html>

8.2 文生图

<!DOCTYPE html>

<head>

maximum-scale=1, minimum-scale=1, width=device-width">

</head>

<div >

<el-form :inline="true">

<el-form-item label="问题">

<el-input v-model="imgtext" placeholder="请输入问题"></el-input>

</el-form-item>

<el-form-item>

<el-button type="primary" @click="fetchAnswer">提问</el-button>

</el-form-item>

</el-form>

</div>

</body>

new Vue({

el: "#app", data: { imgtext: '', imgresponse: '' },

methods: {

fetchAnswer: function() { //生成图片

let vm = this;

axios.get("http://localhost:9320/getImgUrl?prompt="+ this.imgtext).then(function(response) {

vm.imgresponse = response.data

}).catch(function(error) {

console.log(error);

})

}

})

</script>

</html>

8.3 文生语音合成

<!DOCTYPE html>

<head>

maximum-scale=1, minimum-scale=1, width=device-width">

</head>

<div >

<h1 >文字转语音</h1>

<el-form :inline="true">

<el-form-item label="问题">

<el-input v-model="mp3text" placeholder="请输入问题"></el-input>

</el-form-item>

<el-form-item>

<el-button type="primary" @click="Mp3Answer">提问</el-button>

</el-form-item>

</el-form>

</div>

</div>

</body>

new Vue({

el: "#app", data: { mp3text:'', mp3response:'', mp3text2response:'' },

methods: {

Mp3Answer: function() { //文字转语音

let vm = this

axios({

url:'http://localhost:9320/speech?prompt='+this.mp3text, responseType:'blob'

}).then(result=>{

vm.mp3response = window.URL.createObjectURL(result.data)

})

axios({ //文字输出结果

url:'http://localhost:9320/cvtMp3?message='+this.mp3text, responseType:'blob'

}).then(result=>{

vm.mp3text2response = window.URL.createObjectURL(result.data)

})

}

})

</script>

</html>

9 浏览器运行交互测试

9.1 文生问答

9.2 文生图

9.3 文生语音合成

参考

快速开发生成式JavaAI应用--https://sca.aliyun.com/?spm=0.29160081.0.0.282a291fH6ibeR

SpringAI使用通义千问的具体步骤和方法--https://sca.aliyun.com/blog/faq/sca-user-question-history15328/

阿里出击SpringCloudAlibabaAI初体验，https://blog.csdn.net/m0_63171455/article/details/140976640，2024.8.7

10分钟接入AI大模型SpringCloudAlibaba，https://blog.csdn.net/qq_17153885/article/details/140835601，2024.8.1

让你快速入门SpringCloudAlibabaAI，https://blog.csdn.net/qq_34742146/article/details/140503128，2024.7.29

阿里也出手了SpringCloudAlibabaAI问世，https://blog.csdn.net/rong09_13/article/details/139723097，2024.6.26

AI框架之SpringAI与SpringCloudAlibabaAI使用讲解，https://blog.csdn.net/u012060033/article/details/139461527，2024.6.6

spring-cloud-starter-alibaba-ai无法引入如何处理--https://sca.aliyun.com/blog/faq/sca-user-question-history15816/