鸿蒙开发系列 · 专家篇
前言
前几篇我们构建了一个功能完整的语音备忘录应用：录音、播放、定位、搜索、账号登录、会员购买。但当我们审视这个产品的竞争力时，会发现一个尴尬的事实——市面上同类应用成百上千，用户为什么要选你？
答案是AI。如果用户的录音能自动转写成文字，如果应用能智能提取待办事项，如果能根据录音内容自动分类标签——这些能力就能形成差异化壁垒。过去，实现这些需要调用云端大模型API，意味着网络依赖、延迟、成本和隐私风险。但现在，端侧AI正在改变这个等式。
HarmonyOS提供的HiAI Foundation Kit让开发者可以在设备本地运行AI模型，无需联网即可完成图像识别、语音识别、文本理解等任务。本文将带你理解端侧AI的技术原理，并实战集成NLP和语音能力。
一、端侧AI的技术现状与价值
1.1 端侧AI vs 云端AI
端侧AI不是云端AI的替代，而是互补。敏感数据本地处理，复杂任务云端协同，这才是合理的架构。
1.2 HiAI Foundation Kit 能力矩阵
HiAI Foundation Kit提供的能力分为几个大类：
对于我们的语音备忘录应用，最有价值的是语音转文字（ASR）和文本理解（NLP）。
二、语音转文字：ASR 实战
2.1 ASR 架构与流程
┌──────────────┐ 音频流 ┌──────────────┐ 特征提取 ┌──────────────┐
│ 麦克风输入 │ ─────────→ │ 音频预处理 │ ───────────→ │ 编码器模型 │
│ │ │ 降噪/VAD/分帧 │ │ (端侧运行) │
└──────────────┘ └──────────────┘ └───────┬──────┘
│
▼
┌──────────────┐ 文本结果 ┌──────────────┐ 解码 ┌──────────────┐
│ 应用展示 │ ←─────────── │ 后处理 │ ←──────── │ 解码器模型 │
│ │ 标点/格式化 │ 标点/人名识别 │ │ (端侧运行) │
└──────────────┘ └──────────────┘ └──────────────┘
2.2 语音识别接入
HarmonyOS的ASR能力通过 @kit.ASRKit 提供：
// entry/src/main/ets/utils/AsrEngine.ets
import { speechRecognizer } from '@kit.ASRKit';
import { BusinessError } from '@kit.BasicServicesKit';
export enum AsrState {
IDLE = 'idle',
INITIALIZING = 'initializing',
LISTENING = 'listening',
RECOGNIZING = 'recognizing',
COMPLETED = 'completed',
ERROR = 'error'
}
export class AsrEngine {
private engine: speechRecognizer.SpeechRecognitionEngine | null = null;
@State state: AsrState = AsrState.IDLE;
private callbacks: {
onResult?: (text: string, isFinal: boolean) => void;
onError?: (code: number, msg: string) => void;
onStateChange?: (state: AsrState) => void;
} = {};
setCallbacks(callbacks: typeof this.callbacks) {
this.callbacks = callbacks;
}
private setState(state: AsrState) {
this.state = state;
this.callbacks.onStateChange?.(state);
}
// 初始化引擎
async initialize(): Promise<void> {
try {
this.setState(AsrState.INITIALIZING);
const initParams: speechRecognizer.EngineCreateParam = {
context: getContext(),
locale: 'zh-CN',
// 是否开启离线识别
enableOffline: true
};
this.engine = await speechRecognizer.createSpeechRecognizer(initParams);
this.setupEngineCallbacks();
this.setState(AsrState.IDLE);
} catch (err) {
const error = err as BusinessError;
console.error('ASR引擎初始化失败:', error);
this.setState(AsrState.ERROR);
throw err;
}
}
private setupEngineCallbacks() {
if (!this.engine) return;
// 识别结果回调（可能多次触发，isFinal=false是中间结果）
this.engine.on('result', (result: speechRecognizer.SpeechRecognitionResult) => {
const text = result.result;
const isFinal = result.isLast;
console.info(`识别结果: ${text}, 是否最终: ${isFinal}`);
this.callbacks.onResult?.(text, isFinal);
if (isFinal) {
this.setState(AsrState.COMPLETED);
} else {
this.setState(AsrState.RECOGNIZING);
}
});
// 错误回调
this.engine.on('error', (error: BusinessError) => {
console.error('ASR错误:', error.code, error.message);
this.callbacks.onError?.(error.code, error.message);
this.setState(AsrState.ERROR);
});
// 开始监听回调
this.engine.on('start', () => {
console.info('开始监听');
this.setState(AsrState.LISTENING);
});
// 完成回调
this.engine.on('complete', () => {
console.info('识别完成');
this.setState(AsrState.COMPLETED);
});
}
// 开始识别（实时流式）
async startListening(): Promise<void> {
if (!this.engine) throw new Error('引擎未初始化');
const params: speechRecognizer.StartParams = {
// 识别场景
scene: speechRecognizer.SceneType.SCENE_CHAT,
// 是否开启标点
enablePunctuation: true,
// 是否开启数字智能转换
enableNumberConvert: true
};
await this.engine.startListening(params);
}
// 停止识别
async stopListening(): Promise<void> {
if (!this.engine) return;
await this.engine.stopListening();
}
// 取消识别
async cancel(): Promise<void> {
if (!this.engine) return;
await this.engine.cancel();
this.setState(AsrState.IDLE);
}
// 释放引擎
async release(): Promise<void> {
if (this.engine) {
await this.engine.destroy();
this.engine = null;
}
this.setState(AsrState.IDLE);
}
}
2.3 录音文件转文字（非实时）
对于已有的录音文件，可以使用文件识别模式：
// 文件转文字
async transcribeAudioFile(filePath: string): Promise<string> {
if (!this.engine) throw new Error('引擎未初始化');
return new Promise((resolve, reject) => {
let finalText = '';
// 临时设置回调收集结果
const originalCallback = this.callbacks.onResult;
this.callbacks.onResult = (text, isFinal) => {
if (isFinal) {
finalText = text;
this.callbacks.onResult = originalCallback;
resolve(finalText);
}
};
this.engine?.startListening({
audioSource: {
type: speechRecognizer.AudioSourceType.FILE,
filePath: filePath
},
enablePunctuation: true
}).catch(reject);
});
}
2.4 语音转写页面
// entry/src/main/ets/pages/TranscribePage.ets
import { AsrEngine, AsrState } from '../utils/AsrEngine';
@Entry
@Component
struct TranscribePage {
private asrEngine: AsrEngine = new AsrEngine();
@State state: AsrState = AsrState.IDLE;
@State transcript: string = '';
@State interimText: string = '';
aboutToAppear() {
this.asrEngine.setCallbacks({
onResult: (text, isFinal) => {
if (isFinal) {
this.transcript += (this.transcript ? '\n' : '') + text;
this.interimText = '';
} else {
this.interimText = text;
}
},
onStateChange: (state) => this.state = state,
onError: (code, msg) => {
console.error(`ASR错误 ${code}: ${msg}`);
}
});
this.asrEngine.initialize();
}
aboutToDisappear() {
this.asrEngine.release();
}
private async toggleListening() {
if (this.state === AsrState.LISTENING || this.state === AsrState.RECOGNIZING) {
await this.asrEngine.stopListening();
} else {
this.interimText = '';
await this.asrEngine.startListening();
}
}
build() {
Column({ space: 16 }) {
Text('语音转文字')
.fontSize(24)
.fontWeight(FontWeight.Bold)
.margin({ top: 32 })
// 转写结果区域
Column() {
Scroll() {
Column({ space: 8 }) {
Text(this.transcript)
.fontSize(16)
.width('100%')
.lineHeight(24)
if (this.interimText) {
Text(this.interimText)
.fontSize(16)
.fontColor('#2196F3')
.width('100%')
.lineHeight(24)
}
}
.width('100%')
.padding(16)
}
.width('100%')
.layoutWeight(1)
}
.width('100%')
.layoutWeight(1)
.backgroundColor('#F5F5F5')
.borderRadius(12)
.margin(16)
// 状态指示
Text(this.getStateText())
.fontSize(14)
.fontColor(this.state === AsrState.LISTENING ? '#F44336' : '#888')
// 控制按钮
Button(this.state === AsrState.LISTENING ? '停止' : '开始说话')
.width(160)
.height(56)
.fontSize(18)
.backgroundColor(this.state === AsrState.LISTENING ? '#F44336' : '#4CAF50')
.onClick(() => this.toggleListening())
.margin({ bottom: 32 })
}
.width('100%')
.height('100%')
.backgroundColor('#FFFFFF')
}
private getStateText(): string {
switch (this.state) {
case AsrState.IDLE: return '点击开始说话';
case AsrState.LISTENING: return '正在聆听...';
case AsrState.RECOGNIZING: return '识别中...';
case AsrState.COMPLETED: return '识别完成';
case AsrState.ERROR: return '识别出错';
default: return '';
}
}
}
三、NLP 文本理解能力
3.1 HiAI NLP 能力介绍
HiAI Foundation Kit的NLP模块提供以下能力：
3.2 文本分类与实体识别
// entry/src/main/ets/utils/NlpEngine.ets
import { hiai } from '@kit.HiAIFoundationKit';
export interface ClassificationResult {
label: string;
confidence: number;
}
export interface Entity {
text: string;
type: string; // PERSON / LOCATION / ORGANIZATION / TIME / etc.
start: number;
end: number;
}
export class NlpEngine {
private textClassifier: hiai.TextClassifier | null = null;
private entityExtractor: hiai.EntityExtractor | null = null;
async initialize(): Promise<void> {
// 初始化文本分类器
this.textClassifier = await hiai.createTextClassifier({
context: getContext(),
modelType: hiai.TextClassifierType.GENERAL
});
// 初始化实体提取器
this.entityExtractor = await hiai.createEntityExtractor({
context: getContext()
});
}
// 文本分类
async classify(text: string): Promise<ClassificationResult[]> {
if (!this.textClassifier) throw new Error('分类器未初始化');
const results = await this.textClassifier.classify(text);
return results.map(r => ({
label: r.label,
confidence: r.confidence
}));
}
// 实体识别
async extractEntities(text: string): Promise<Entity[]> {
if (!this.entityExtractor) throw new Error('实体提取器未初始化');
const results = await this.entityExtractor.extract(text);
return results.map(r => ({
text: r.text,
type: r.type,
start: r.startIndex,
end: r.endIndex
}));
}
// 智能提取待办事项
async extractTodos(text: string): Promise<string[]> {
const entities = await this.extractEntities(text);
// 筛选时间相关实体，结合上下文提取待办
const timeEntities = entities.filter(e =>
e.type === 'TIME' || e.type === 'DATE'
);
const todos: string[] = [];
// 简单启发式：包含时间表达式的句子可能是待办
const sentences = text.split(/[。！？\n]/);
sentences.forEach(sentence => {
const hasTime = timeEntities.some(e => sentence.includes(e.text));
const hasAction = /(要|需要|记得|别忘了|必须|应该).+/.test(sentence);
if (hasTime || hasAction) {
todos.push(sentence.trim());
}
});
return todos;
}
// 释放资源
async release(): Promise<void> {
await this.textClassifier?.release();
await this.entityExtractor?.release();
this.textClassifier = null;
this.entityExtractor = null;
}
}
3.3 智能分析录音内容
// entry/src/main/ets/utils/RecordingAnalyzer.ets
import { AsrEngine } from './AsrEngine';
import { NlpEngine, ClassificationResult, Entity } from './NlpEngine';
export interface RecordingAnalysis {
transcript: string;
categories: ClassificationResult[];
entities: Entity[];
todos: string[];
summary: string;
}
export class RecordingAnalyzer {
private asrEngine: AsrEngine;
private nlpEngine: NlpEngine;
constructor() {
this.asrEngine = new AsrEngine();
this.nlpEngine = new NlpEngine();
}
async initialize(): Promise<void> {
await Promise.all([
this.asrEngine.initialize(),
this.nlpEngine.initialize()
]);
}
// 分析录音文件
async analyzeRecording(filePath: string): Promise<RecordingAnalysis> {
// 1. 语音转文字
const transcript = await this.asrEngine.transcribeAudioFile(filePath);
// 2. NLP分析
const [categories, entities, todos] = await Promise.all([
this.nlpEngine.classify(transcript),
this.nlpEngine.extractEntities(transcript),
this.nlpEngine.extractTodos(transcript)
]);
// 3. 生成摘要（简单版：取前50字）
const summary = transcript.length > 50
? transcript.substring(0, 50) + '...'
: transcript;
return {
transcript,
categories,
entities,
todos,
summary
};
}
async release(): Promise<void> {
await Promise.all([
this.asrEngine.release(),
this.nlpEngine.release()
]);
}
}
四、端侧AI的性能与优化
4.1 模型加载策略
端侧AI模型通常占用数十到数百MB存储，加载到内存后又占用大量RAM。优化策略：
// 懒加载 + 超时释放的封装
export class LazyAiEngine<T> {
private engine: T | null = null;
private factory: () => Promise<T>;
private releaseFn: (engine: T) => Promise<void>;
private idleTimer: number = -1;
private readonly IDLE_TIMEOUT = 5 * 60 * 1000; // 5分钟空闲后释放
constructor(
factory: () => Promise<T>,
releaseFn: (engine: T) => Promise<void>
) {
this.factory = factory;
this.releaseFn = releaseFn;
}
async get(): Promise<T> {
this.resetIdleTimer();
if (!this.engine) {
this.engine = await this.factory();
}
return this.engine;
}
private resetIdleTimer() {
if (this.idleTimer !== -1) {
clearTimeout(this.idleTimer);
}
this.idleTimer = setTimeout(() => {
this.release();
}, this.IDLE_TIMEOUT);
}
private async release() {
if (this.engine) {
await this.releaseFn(this.engine);
this.engine = null;
}
}
}
4.2 端侧AI的局限与应对
五、小结与下篇预告
本文带你进入了HarmonyOS的AI能力域：
端侧AI定位：延迟低、隐私好、成本低，与云端AI互补
ASR语音识别：实时流式识别、文件转写、状态管理
NLP文本理解：文本分类、实体识别、智能待办提取
性能优化：懒加载、量化模型、空闲释放
端侧AI让我们的语音备忘录从"录音工具"升级为"智能语音助手"——自动转写、智能分类、待办提取，这些能力构成了产品的差异化壁垒。
但鸿蒙的野心不止于单设备。HarmonyOS的核心理念是"万物互联"，而ArkUI-X正是这一理念的延伸——一套ArkTS代码，同时运行在鸿蒙手机、Android、iOS甚至Web上。第18篇将深入ArkUI-X跨平台开发，讲解工程结构、平台适配、条件编译和发布策略，让你的应用真正突破设备边界。
参考阅读
本系列第16篇：《支付与账号集成：华为账号、应用内支付接入指南》