340 lines
7.4 KiB
Markdown
340 lines
7.4 KiB
Markdown
# 高可用集群部署手册
|
|
|
|
## 一、环境准备
|
|
|
|
### 1.1 虚拟机配置要求
|
|
|
|
| 节点 | IP地址 | CPU | 内存 | 磁盘 | 操作系统 |
|
|
|------|--------|-----|------|------|----------|
|
|
| Node-01 | 192.168.1.101 | 4核 | 8GB | 100GB | Ubuntu 22.04 / CentOS 8 |
|
|
| Node-02 | 192.168.1.102 | 4核 | 8GB | 100GB | Ubuntu 22.04 / CentOS 8 |
|
|
| Node-03 | 192.168.1.103 | 4核 | 8GB | 100GB | Ubuntu 22.04 / CentOS 8 |
|
|
|
|
### 1.2 软件依赖
|
|
|
|
每台虚拟机需要安装以下软件：
|
|
|
|
```bash
|
|
# Ubuntu
|
|
apt-get update
|
|
apt-get install -y docker.io docker-compose-plugin openjdk-17-jdk maven git curl wget
|
|
|
|
# CentOS
|
|
yum install -y docker docker-compose-plugin java-17-openjdk-devel maven git curl wget
|
|
|
|
# 启动Docker
|
|
systemctl start docker
|
|
systemctl enable docker
|
|
|
|
# 添加当前用户到docker组
|
|
usermod -aG docker $USER
|
|
```
|
|
|
|
### 1.3 网络配置
|
|
|
|
确保三台虚拟机之间网络互通：
|
|
|
|
```bash
|
|
# 在每台机器上测试
|
|
ping 192.168.1.101
|
|
ping 192.168.1.102
|
|
ping 192.168.1.103
|
|
|
|
# 开放防火墙端口
|
|
# Ubuntu
|
|
ufw allow 3306,6379,8848,9000,9001,9876,10911,8880,9000:9006/tcp
|
|
|
|
# CentOS
|
|
firewall-cmd --permanent --add-port={3306,6379,8848,9000,9001,9876,10911,8880,9000-9006}/tcp
|
|
firewall-cmd --reload
|
|
```
|
|
|
|
---
|
|
|
|
## 二、部署步骤
|
|
|
|
### 2.1 克隆项目到所有节点
|
|
|
|
```bash
|
|
# 在所有节点执行
|
|
git clone <项目地址> /opt/ha-ovp
|
|
cd /opt/ha-ovp
|
|
```
|
|
|
|
### 2.2 构建项目
|
|
|
|
```bash
|
|
# 在任意节点构建（只需执行一次）
|
|
cd /opt/ha-ovp
|
|
|
|
# 构建common模块
|
|
cd video-common
|
|
mvn clean install -DskipTests
|
|
cd ..
|
|
|
|
# 构建所有服务
|
|
mvn clean package -DskipTests
|
|
|
|
# 构建Sentinel镜像
|
|
cd sentinel-dashboard
|
|
docker build -t sentinel-dashboard:1.8.6 .
|
|
cd ..
|
|
|
|
# 将构建产物分发到其他节点
|
|
scp -r target/ video-*/target/ root@192.168.1.102:/opt/ha-ovp/
|
|
scp -r target/ video-*/target/ root@192.168.1.103:/opt/ha-ovp/
|
|
```
|
|
|
|
### 2.3 初始化各节点
|
|
|
|
```bash
|
|
# Node-01 (192.168.1.101)
|
|
chmod +x deploy-ha.sh
|
|
./deploy-ha.sh 1 init
|
|
|
|
# Node-02 (192.168.1.102)
|
|
chmod +x deploy-ha.sh
|
|
./deploy-ha.sh 2 init
|
|
|
|
# Node-03 (192.168.1.103)
|
|
chmod +x deploy-ha.sh
|
|
./deploy-ha.sh 3 init
|
|
```
|
|
|
|
### 2.4 按顺序启动服务
|
|
|
|
#### 步骤1: 启动基础设施 (所有节点)
|
|
|
|
```bash
|
|
# Node-01 先启动
|
|
./deploy-ha.sh 1 infra
|
|
|
|
# 等待Node-01完全启动后 (约2分钟)
|
|
# Node-02 启动
|
|
./deploy-ha.sh 2 infra
|
|
|
|
# Node-03 启动
|
|
./deploy-ha.sh 3 infra
|
|
```
|
|
|
|
#### 步骤2: 配置MySQL主从复制
|
|
|
|
```bash
|
|
# Node-01 查看主节点状态
|
|
docker exec mysql mysql -uroot -proot123456 -e "SHOW MASTER STATUS\G"
|
|
|
|
# 记录 File 和 Position 值
|
|
|
|
# Node-02 配置从节点
|
|
docker exec mysql mysql -uroot -proot123456 <<EOF
|
|
CHANGE MASTER TO
|
|
MASTER_HOST='192.168.1.101',
|
|
MASTER_USER='repl',
|
|
MASTER_PASSWORD='repl_password_2024',
|
|
MASTER_LOG_FILE='mysql-bin.000001',
|
|
MASTER_LOG_POS=<Position值>;
|
|
START SLAVE;
|
|
SHOW SLAVE STATUS\G
|
|
EOF
|
|
|
|
# Node-03 配置从节点 (同上)
|
|
```
|
|
|
|
#### 步骤3: 配置Redis主从复制
|
|
|
|
```bash
|
|
# Node-02 和 Node-03 执行
|
|
docker exec redis redis-cli -a redis_HA_Cluster_2024 REPLICAOF 192.168.1.101 6379
|
|
|
|
# 验证
|
|
docker exec redis redis-cli -a redis_HA_Cluster_2024 INFO replication
|
|
```
|
|
|
|
#### 步骤4: 配置Nacos集群
|
|
|
|
访问 http://192.168.1.101:8848/nacos (账号: nacos/nacos)
|
|
|
|
创建命名空间 `public`，并添加以下配置文件：
|
|
|
|
| Data ID | Group | 说明 |
|
|
|---------|-------|------|
|
|
| video-gateway.yaml | DEFAULT_GROUP | 网关配置 |
|
|
| video-auth.yaml | DEFAULT_GROUP | 认证服务配置 |
|
|
| video-user.yaml | DEFAULT_GROUP | 用户服务配置 |
|
|
| video-content.yaml | DEFAULT_GROUP | 内容服务配置 |
|
|
| video-comment.yaml | DEFAULT_GROUP | 评论服务配置 |
|
|
| video-recommend.yaml | DEFAULT_GROUP | 推荐服务配置 |
|
|
| video-admin.yaml | DEFAULT_GROUP | 管理后台配置 |
|
|
|
|
#### 步骤5: 初始化MinIO
|
|
|
|
```bash
|
|
# 任意节点执行
|
|
docker run --rm --network ha-network minio/mc <<EOF
|
|
mc alias set myminio http://192.168.1.101:9000 minioadmin minioadmin123
|
|
mc mb myminio/video-platform --ignore-existing
|
|
mc anonymous set download myminio/video-platform
|
|
EOF
|
|
```
|
|
|
|
#### 步骤6: 启动微服务
|
|
|
|
```bash
|
|
# Node-01 启动微服务
|
|
./deploy-ha.sh 1 services
|
|
|
|
# Node-02 启动微服务
|
|
./deploy-ha.sh 2 services
|
|
|
|
# Node-03 不启动微服务（仅作为基础设施备份）
|
|
```
|
|
|
|
#### 步骤7: 启动Nginx负载均衡
|
|
|
|
```bash
|
|
# Node-01 启动Nginx
|
|
./deploy-ha.sh 1 start-nginx
|
|
```
|
|
|
|
---
|
|
|
|
## 三、验证部署
|
|
|
|
### 3.1 检查服务状态
|
|
|
|
```bash
|
|
# 各节点执行
|
|
./deploy-ha.sh <node-id> status
|
|
|
|
# 检查Nacos服务注册
|
|
curl http://192.168.1.101:8848/nacos/v1/ns/instance/list?serviceName=video-gateway
|
|
|
|
# 检查Redis主从
|
|
docker exec redis redis-cli -a redis_HA_Cluster_2024 INFO replication
|
|
|
|
# 检查MySQL主从
|
|
docker exec mysql mysql -uroot -proot123456 -e "SHOW SLAVE STATUS\G"
|
|
```
|
|
|
|
### 3.2 访问服务
|
|
|
|
| 服务 | 地址 |
|
|
|------|------|
|
|
| Nacos控制台 | http://192.168.1.101:8848/nacos |
|
|
| Sentinel控制台 | http://192.168.1.101:8880 |
|
|
| MinIO控制台 | http://192.168.1.101:9101 |
|
|
| RocketMQ控制台 | http://192.168.1.102:8080 (需额外部署) |
|
|
| API网关 | http://192.168.1.100 (通过Nginx) |
|
|
|
|
### 3.3 高可用测试
|
|
|
|
#### 测试服务故障转移
|
|
|
|
```bash
|
|
# 停止Node-01上的网关服务
|
|
docker stop video-gateway
|
|
|
|
# 访问API验证（应该自动切换到Node-02）
|
|
curl http://192.168.1.100/auth/captcha
|
|
|
|
# 恢复服务
|
|
docker start video-gateway
|
|
```
|
|
|
|
#### 测试Redis主从切换
|
|
|
|
```bash
|
|
# 停止Node-01的Redis
|
|
docker stop redis
|
|
|
|
# Sentinel会自动将从节点提升为主节点
|
|
# 查看Sentinel日志
|
|
docker logs redis-sentinel
|
|
|
|
# 恢复Redis
|
|
docker start redis
|
|
```
|
|
|
|
#### 测试MySQL主从切换
|
|
|
|
```bash
|
|
# 停止主节点MySQL
|
|
docker stop mysql
|
|
|
|
# 查看从节点状态
|
|
docker exec mysql mysql -uroot -proot123456 -e "SHOW SLAVE STATUS\G"
|
|
```
|
|
|
|
---
|
|
|
|
## 四、常见问题
|
|
|
|
### Q1: Nacos集群无法启动
|
|
|
|
检查各节点时间是否同步：
|
|
|
|
```bash
|
|
# 同步时间
|
|
apt-get install -y ntpdate
|
|
ntpdate pool.ntp.org
|
|
```
|
|
|
|
### Q2: 服务注册不到Nacos
|
|
|
|
检查网络连通性：
|
|
|
|
```bash
|
|
# 测试Nacos连接
|
|
curl http://192.168.1.101:8848/nacos/v1/console/health/liveness
|
|
```
|
|
|
|
### Q3: Redis连接失败
|
|
|
|
检查Redis密码配置是否一致：
|
|
|
|
```bash
|
|
# 测试连接
|
|
docker exec redis redis-cli -a redis_HA_Cluster_2024 ping
|
|
```
|
|
|
|
### Q4: MySQL主从同步失败
|
|
|
|
```bash
|
|
# 重置从节点
|
|
docker exec mysql mysql -uroot -proot123456 <<EOF
|
|
STOP SLAVE;
|
|
RESET SLAVE ALL;
|
|
CHANGE MASTER TO
|
|
MASTER_HOST='192.168.1.101',
|
|
MASTER_USER='repl',
|
|
MASTER_PASSWORD='repl_password_2024',
|
|
MASTER_LOG_FILE='mysql-bin.000001',
|
|
MASTER_LOG_POS=0;
|
|
START SLAVE;
|
|
EOF
|
|
```
|
|
|
|
---
|
|
|
|
## 五、服务端口汇总
|
|
|
|
| 服务 | Node-01 | Node-02 | Node-03 | 说明 |
|
|
|------|---------|---------|---------|------|
|
|
| MySQL | 3306 | 3306 | 3306 | 数据库主从 |
|
|
| Redis | 6379 | 6379 | 6379 | 缓存主从 |
|
|
| Redis Sentinel | 26379 | 26379 | 26379 | 哨兵 |
|
|
| Nacos | 8848 | 8848 | 8848 | 配置中心集群 |
|
|
| RocketMQ NameServer | 9876 | 9876 | 9876 | NameServer集群 |
|
|
| RocketMQ Broker | - | 10911 | 10911 | Broker双主 |
|
|
| MinIO API | 9100 | 9100 | 9000 | 对象存储API |
|
|
| MinIO Console | 9101 | 9101 | 9001 | 对象存储控制台 |
|
|
| Sentinel Dashboard | 8880 | 8880 | 8880 | 熔断控制台 |
|
|
| video-gateway | 9000 | 9000 | - | API网关 |
|
|
| video-auth | 9001 | 9001 | - | 认证服务 |
|
|
| video-user | 9002 | 9002 | - | 用户服务 |
|
|
| video-content | 9003 | 9003 | - | 内容服务 |
|
|
| video-comment | 9004 | 9004 | - | 评论服务 |
|
|
| video-recommend | 9005 | 9005 | - | 推荐服务 |
|
|
| video-admin | 9006 | 9006 | - | 管理后台 |
|
|
| Nginx | 80 | - | - | 负载均衡 |
|