Recently, I got my hands on the Sipeed Maix4-HAT. I decided to set it up on my Raspberry Pi 5 to run the Qwen2.5-7B-Instruct model for AI inference. Here's a step-by-step guide on how I did it.

Assumptions

• You have a Raspberry Pi 5 with Raspberry Pi OS installed.
• You have connected the Sipeed Maix4-HAT to your Raspberry Pi 5.
• You have a 128GB+ SD card for the OS and model storage.

Driver Setup

Step 1: Update the System

First, I made sure to update my Raspberry Pi 5 to the latest version of Raspberry Pi OS. I ran the following commands in the terminal:

sudo apt update
sudo apt upgrade -y

Step 2: Enable PCIe x1 Interface

Next, enable Raspberry Pi 5's PCIe x1 interface:

sudo raspi-config

Navigate to 6 Advanced Options → A8 PCIe Speed → Select Yes and enable it.

┌──────────────────────────────────────────────────────────────────────────────┐
│  1 System Options       Configure system settings                            │
│  2 Display Options      Configure display settings                           │
│  3 Interface Options    Configure connections to peripherals                 │
│  4 Performance Options  Configure performance settings                       │
│  5 Localisation Options Configure language and regional settings             │
│  6 Advanced Options     Configure advanced settings                          │
│  8 Update               Update this tool to the latest version               │
│  9 About raspi-config   Information about this configuration tool            │
└──────────────────────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────────────────────┐
│  A1 Expand Filesystem       Ensures that all of the SD card is available     │
│  A2 Network Interface Names Enable/disable predictable network i/f names     │
│  A3 Network Proxy Settings  Configure network proxy settings                 │
│  A4 Boot Order              Choose boot device priority (SD/net/USB/NVMe)    │
│  A5 Bootloader Version      Selects the latest or factory default bootloader │
│  A6 Beta Access             Select beta or release software repository       │
│  A7 Wayland                 Switch between X and Wayland backends            │
│  A8 PCIe Speed              Set PCIe x1 port speed                           │
│  A9 Network Install UI      Select display of bootloader network install UI  │
│  A10 Libliftoff             Enable/disable libliftoff hardware overlays      │
│  A11 Shutdown Behaviour     Configure shutdown behavior                      │
└──────────────────────────────────────────────────────────────────────────────┘

After enabling the PCIe x1 interface, reboot your Raspberry Pi 5 to apply the changes.

Step 3: Verify Accelerator Card Detection

After rebooting, verify that the accelerator card is detected with lspci:

$ lspci
0001:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2712 PCIe Bridge (rev 30)
0001:01:00.0 Multimedia video controller: Axera Semiconductor Co., Ltd Device 0650 (rev 01)
0002:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2712 PCIe Bridge (rev 30)
0002:01:00.0 Ethernet controller: Raspberry Pi Ltd RP1 PCIe 2.0 South Bridge

Step 4: Install AXCL Software

After PCIe detection, install the AXCL package for model acceleration. In my case, I was unable to install the driver using sudo apt install axcl_host_aarch64_V3.6.2_20250603154858_NO4873.deb from the provided AIDemos.tar.zst. Instead, I downloaded the deb from AXCL Runtime Releases and installed it with the following commands:

wget https://github.com/AXERA-TECH/axcl-runtime/releases/download/V3.10.2/axcl_host_aarch64_V3.10.2_20251111020143_NO5046.deb
sudo apt install ./axcl_host_aarch64_V3.10.2_20251111020143_NO5046.deb

Reboot the Pi, then verify the installation with axcl-smi:

$ axcl-smi 
+------------------------------------------------------------------------------------------------+
| AXCL-SMI  V3.10.2_20251111020143                                Driver  V3.10.2_20251111020143 |
+-----------------------------------------+--------------+---------------------------------------+
| Card  Name                     Firmware | Bus-Id       |                          Memory-Usage |
| Fan   Temp                Pwr:Usage/Cap | CPU      NPU |                             CMM-Usage |
|=========================================+==============+=======================================|
|    0  AX8850                    V3.10.2 | 0001:01:00.0 |                148 MiB /      945 MiB |
|   --   41C                      -- / -- | 2%        0% |                 18 MiB /     7040 MiB |
+-----------------------------------------+--------------+---------------------------------------+

+------------------------------------------------------------------------------------------------+
| Processes:                                                                                     |
| Card      PID  Process Name                                                   NPU Memory Usage |
|================================================================================================|

Installing Qwen2.5-7B-Instruct

Step 1: Clone the Model Repository

First, install git-lfs if you don't have it:

sudo apt install git-lfs

Then clone the model from Hugging Face:

mkdir ~/Code
cd ~/Code
git clone https://huggingface.co/AXERA-TECH/Qwen2.5-7B-Instruct

Step 2: Set Up Python Environment

Install Miniforge3 for Python environment management:

wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh

Create an env.yml file for the conda environment:

name: llm
channels:
  - conda-forge
dependencies:
  - python=3.12
  - pip
  - pip:
    - transformers==5.2.0
    - jinja2==3.1.6

Install screen for session management (temporary — we will migrate to a systemd service later):

sudo apt install screen

Step 3: Run the Model

Create the conda environment and run the tokenizer:

source ~/.bashrc
conda env create -f env.yml
screen -S llm
conda activate llm
cd ~/Code/Qwen2.5-7B-Instruct/
python qwen2.5_tokenizer_uid.py

In another terminal, execute the following to start the inference:

chmod u+x run_qwen2.5_7b_ctx_int4_axcl_aarch64.sh
chmod u+x main_axcl_aarch64
./run_qwen2.5_7b_ctx_int4_axcl_aarch64.sh

Example output:

prompt >> Hello World
[I][                      SetKVCache][ 614]: prefill_grpid:2 kv_cache_num:128 precompute_len:21 input_num_token:10
[I][                      SetKVCache][ 617]: current prefill_max_token_num:896
[I][                             Run][ 855]: input token num : 10, prefill_split_num : 1
[I][                             Run][ 887]: input_num_token:10
[I][                             Run][1016]: ttft: 930.97 ms
Hello! Welcome. Is there anything specific you'd like to know or discuss today?

[N][                             Run][1168]: hit eos,avg 4.26 token/s

prompt >> Who are you
[I][                      SetKVCache][ 614]: prefill_grpid:2 kv_cache_num:128 precompute_len:48 input_num_token:11
[I][                      SetKVCache][ 617]: current prefill_max_token_num:896
[I][                             Run][ 855]: input token num : 11, prefill_split_num : 1
[I][                             Run][ 887]: input_num_token:11
[I][                             Run][1016]: ttft: 929.71 ms
I'm Qwen, a large language model created by Alibaba Cloud. How can I assist you today?

[N][                             Run][1168]: hit eos,avg 4.30 token/s

Conclusion

The Sipeed Maix4-HAT with the AX650N NPU delivers approximately 4.3 tokens/second for Qwen2.5-7B-Instruct inference on the Raspberry Pi 5. While not blazingly fast, it's impressive for a low-power edge device and suitable for local AI experimentation.

References

• Sipeed Maix4-HAT Wiki
• AXCL Runtime Releases
• Qwen2.5-7B-Instruct on Hugging Face

When installing openclaw, it is important to ensure that the host machine is properly secured to prevent unauthorized access and potential vulnerabilities. Here are some steps to enhance the security of the host machine when installing openclaw:

Install on a Dedicated Machine

Use a dedicated machine for openclaw installation to minimize the attack surface and isolate it from other services.

Install fail2ban

On my raspberry pi, I installed fail2ban to protect against brute-force attacks. You can install it using the following commands:

sudo apt-get update
sudo apt-get install fail2ban
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
sudo systemctl status fail2ban

Restrict Telegram Bot Access

If you are using a Telegram bot with openclaw, ensure that the bot is configured to only accept messages from authorized users. e.g.

$ cat ~/.openclaw/openclaw.json
    ...
    "telegram": {
      "enabled": true,
      "dmPolicy": "allowlist",
      "botToken": "<YOUR_BOT_TOKEN>",
      "allowFrom": [
        "<YOUR_TELEGRAM_USER_ID>"
      ],
      "groupPolicy": "allowlist",
      "streamMode": "partial"
    }
    ...

Regular Updates

Keep the host machine's operating system and all installed packages up to date with the latest security patches.

On the ubuntu system, we can also enable automatic security updates by installing the unattended-upgrades package:

sudo apt-get install unattended-upgrades
sudo dpkg-reconfigure unattended-upgrades

Run security Audits

Regularly run security audits using tools like Lynis or OpenVAS to identify and address potential vulnerabilities.

openclaw security audit --deep

For the VPS installation, you may also want to have a look on the X post: https://x.com/BitsagaRob/status/2015757134760202469

終於把網站遷移完成，使用了自己最近比較熟悉的 VuePress。沒想到，8 年後，在 AI Agent 的協助下，幫我提交 pull request，在我需要極少的人為 coding 介入下，完成了這個任務。而這個 post 也是透過 send message 給 agent 的形式發佈的呢！

遷移歷程

這次遷移經歷了一些波折：

1. 初期嘗試 VitePress
   最開始想用 VitePress（Vue 3 + Vite），但遇到了一些配置問題。

2. 回退到 VuePress v1
   嘗試使用 VuePress v1，但發現 Vue 2 已經 EOL，而且有 105 個安全漏洞。

3. 最終升級到 VuePress v2

   • 從 Vue 2 升級到 Vue 3
   • 安全漏洞：105 個 → 0 個
   • Build 時間：約 36 秒
   • 使用 Vite bundler，速度快很多

AI Agent 的幫助

整個過程中，AI Agent 幫我：

• 自動處理 Git 操作（fetch、merge、commit、push）
• 修復 package-lock.json 問題
• 解決 GitHub Actions 部署配置
• 修復文章中的語法問題（例如 <script> 標籤導致的 webpack 錯誤）
• 甚至幫我寫這篇文章並發 PR！

技術細節

• 框架： VuePress v2.0.0-rc.26
• Vue 版本： 3.5.13
• Bundler： Vite
• 部署： GitHub Actions → GitHub Pages
• 文章數： 34 篇（從 2014 年至今）

感想

科技進步真的很快。8 年前寫第一篇 blog 時，還在手動寫 HTML、CSS。現在只需要跟 AI 對話，它就能幫我完成大部分的技術工作。這篇文章本身就是最好的證明——我只是用自然語言告訴 AI 我想寫什麼，它就幫我整理好格式、創建 PR，甚至處理了 Git 操作。

期待未來會有更多這樣的自動化工具！🚀

輸入以下指令

vi  /etc/yum.repos.d/google.repo

貼上以下內容

[google64]
name=Google - x86_64
baseurl=http://dl.google.com/linux/rpm/stable/x86_64
enabled=1
gpgcheck=1
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub

之後再輸入

sudo yum install -y google-chrome-stable

若你只有windows 或 mac os 版本的Cisco AnyConnect Secure Mobility Client，而想在Centos 下使用和Cisco AnyConnect Secure Mobility Client功能相同的軟件， 可以考慮安裝openconnet

sudo yum install openconnect

使用方式:

sudo openconnect <vpn ip>

輸入username + password即可

留意到resolv.conf 更新了

sudo cat /etc/resolv.conf

按ctrl + c 即可斷開VPN連接

這半年都沒有把自己遇到的IT問題更新在這個blog呢，看來還是需要加把勁更新一下個blog呢。不進則退。

這半年接主要觸到的而掌握技術有 ansible 和 vue.js + nuxt + webpack + es6 等等，但做了web developer這麼耐，居然還未在這個網頁有好好整理自己曾經開過而闗閉了的網站，還有希望在不久的未來，建立一個多人使用的高質素網站呢!畢竟我是一個developer，沒有屬於自己的網站也不太合理吧。

近期對Latex相當有興趣， 之前曾經稍為在Wiki了解過Latex，覺得Symbol好像看起來好像好複雜，所以便提不起興趣學了。 直至近期發現好多整齊漂亮的pdf文件原來都是由Latex的編譯器建立，便開始想了解一下，當學習一個全新的markup language。 以下的範例會建立一個pdf文件，簡單地記綠「Today I Learned」，好像https://github.com/thoughtbot/til 或 https://github.com/jbranchaud/til，只是用Latex編寫而已，雖然這個情況或者用markdown會比較合適，但是用來練習Latex也不錯:

首先安裝 texlive:

sudo apt-get install texlive-full

Text Editor 我覺得用atom便好了無需再額多安裝 texmaker。反正本身電腦只使用了128G SSD， 沒有太多空間...

之後把以下原始碼儲存為main.tex，不妨花點時間了解一下latex語法，推薦不妨抽少少時間看看Latex in 157 minutes

\documentclass{article}

\usepackage{hyperref}

\RequirePackage{fontspec}
\RequirePackage{xeCJK}

\setCJKmainfont{Droid Sans Fallback}
\XeTeXlinebreaklocale "zh"
\XeTeXlinebreakskip = 0pt plus 1pt

\title{Today I learned.今日我所學}
\author{Nick Shek}
\date{2017-02-07}

\begin{document}

\maketitle

\clearpage

\tableofcontents

\clearpage

\section{MySQL}

\begin{itemize}
\item Setting up MySQL replication without the downtime: \href{https://plusbryan.com/mysql-replication-without-downtime}{https://plusbryan.com/mysql-replication-without-downtime}
\end{itemize}

\section{Latex}

\begin{itemize}
\item Latex 中文化解決方案:
  \begin{itemize}
    \item \href{https://goo.gl/6ajvrM}{https://goo.gl/6ajvrM}
    \item \href{http://pangomi.blogspot.hk/2012/11/latex-in-ubuntu.html}{http://pangomi.blogspot.hk/2012/11/latex-in-ubuntu.html}
  \end{itemize}
\item Create Table of contents in Latex \href{https://www.sharelatex.com/learn/Table\_of\_contents}{https://www.sharelatex.com/learn/Table\_of\_contents}
\end{itemize}





\end{document}

確保Droid Sans Fallback字型已經安裝後，執行

xelatex main.tex

如無意外，便能成功建立一個pdf，及中文字型能正確顯示。

參考了不同網站後，現在總算能輕鬆建立pdf，而pdf本身能顯示中文。 了解Latex基本Symbol後，其實建立一份Latex 沒有想像中那麼複雜。

如有興趣，也不妨看看: https://github.com/nickshek/latex-til

有時間的話會不定期更新

Source: http://milq.github.io/install-latex-ubuntu-debian/

Ansible在前幾年是一個非常熱門的開源項目，以我理解ansible其中一個功能是可用來部署應用程式，但是我個人覺得部署應用程式暫時有 capistrano或deployer-php也非常足夠，同時部署應用程式於多個伺服器也勝任有餘，總比手動部署來得有效率。

因此比起用Ansible來去試一試寫一個部署scripts，我想使用Ansible試一試一個簡單的例子去在各server 執行一些command (e.g df -h)，再把結果以電子郵件傳送，這樣也可以感受一下Ansible的威力，雖然個人比起編寫yaml (yaml 內或text template 可以使用 jinja2 template language，一個類似Django template engine的template engine)，個人更喜愛寫Ruby 或 PHP scripts 。

首先在/etc/ansible/ansible.cfg 設定 host_key_checking = False，這樣會令測試ansible時簡單一點，ansible.cfg 的範例如下所示:

[defaults]
host_key_checking = False
log_path=/path/to/log

之後建立一個ansible playbook e.g site.yml

- hosts: all:!local
  tasks:
  - name: ping all hosts
    ping:
    register: ping_result
    ignore_errors: yes

  - name: check disk space for all hosts
    command: df -h
    register: disk_result
    ignore_errors: yes

  - name: check high cpu usage process
    shell: ps -eo pcpu,user,args | sort -r -k1 | head -n5
    register: cpu_result
    ignore_errors: yes

  - name: run ifconfig
    shell: ifconfig
    register: ifconfig_result
    ignore_errors: yes

  - name: template email
    local_action: template src=./files/email_template.j2 dest='./tmp/{{ hostvars[inventory_hostname]["inventory_hostname"] }}.txt'

- hosts: local
  tasks:
  - name: create result email
    template: src=./files/email_summary_template.j2 dest=./tmp/email.txt
    delegate_to: localhost

  - name: send email result
    local_action: mail host='{{ email_host }}' port='{{ email_port }}' to='{{ email_to }}' username='{{ email_username }}' password='{{ email_password }}' subject='Report' body='{{ lookup("file", "./tmp/email.txt") }}' from='{{ email_from }}' charset='UTF-8' subtype='html'

以上的yaml script的流程好簡單，純粹用ping + shell module在不同的server 去執行不同的指令，之後用register 去把執行結果紀錄至指定的variable。因為萬一執行時出現錯誤，我們還是希望ansible playbook可繼續執行，因此設定ignore_errors = true即可。

下一步便是把各servers的執行結果變為一個template，因為這個執行結果在local machine的下一個tasks是需要的，所以使用了local_action

以下便用 email_template.j2 的原始碼:

{% if ping_result is defined %}
<h2>Ping Command Result</h2>
-----------------------------------<br/>
{{ ping_result | to_nice_yaml | replace("\n","
") }}
<br/>
{% endif %}

{% if disk_result is defined %}
<h2>Disk Command Result</h2>
-----------------------------------<br/>
{{ disk_result | to_nice_yaml | replace("\n","
") }}
<br/>
{% endif %}

{% if cpu_result is defined %}
<h2>Command result for ps -eo pcpu,user,args | sort -r -k1 | head -n5</h2>
-----------------------------------<br/>
{{ cpu_result | to_nice_yaml | replace("\n","
") }}
{% endif %}
<br/>

執行結果是一個python 的 dict，為了確保所有的內容都被列印出來，純粹用to_nice_yaml 列印出來即可。 一下步便是把每一個inventory的執行結果整理至一個email template，然後寄出。email_summary_template.j2的內容如下所示:

{% for host in groups['all'] %}
{% if host != 'localhost' %}
<h1>Result for {{ hostvars[host]['inventory_hostname'] }}</h1>
<b>==================================================================</b><br/>
{{lookup("file", "./tmp/" + hostvars[host]["inventory_hostname"] + ".txt")}}
{% endif %}
{% endfor %}

請注意，send email result的這個tasks全部都使用了variable，避免直接寫死任何容易更改設定在 playboook內，這一個動作明顯不符合Ansible本身的哲學，variable可以放在group_vars/all :

---
email_host: <email_host>
email_port: <email_port>
email_to: <email_to>
email_username: <email_username>
email_password: <email_password>
email_from: <email_from>

ansible的建議資料夾結構請參考 http://docs.ansible.com/ansible/playbooks_best_practices.html#directory-layout

最後我們還是編寫hosts 檔案，以下是hosts的檔案範例，建議先設定ssh key來做認證，小心保管好private key + public key，避免把密碼寫在hosts file

[local]
localhost ansible_connection=local

[servers]
<example1.com> ansible_user=<example_user1> ansible_port=<example_port1> ansible_ssh_private_key_file=.ssh/id_rsa
<example1.com> ansible_user=<example_user2> ansible_port=<example_port2> ansible_ssh_private_key_file=.ssh/id_rsa

設定方法可參考 https://blog.longwin.com.tw/2005/12/ssh_keygen_no_passwd/

執行ansible playbook的方法相當簡單，如下所示:

ansible-playbook -i hosts site.yml

這樣我們便完成了一個好簡單的ansible playbook，這個playbook也有相當多的改進的地方，例如我們都把tasks 都放在site.yml，現在還好，沒有太多tasks。但沒有按roles去把不同的tasks放入去長遠都會有維護上的問題，site.yml愈簡單愈容易維護，另外一個可以改進的地方是lookup時的錯誤處理。

想知道這個project的檔案結構，可參考 https://github.com/nickshek/ansible_mail_example

怕自己忘記了，放上這個blog。這個指令在開發Docker Image 時還是經常使用的。

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)

作為一個偏好使用Linux平台的Developer， 或會經常遇到一次性命名多個File的問題。 如果File數目不算多的話，還可以透過mv command 一一執行。 但是如果遇上成千隻呢?那肯定要想一個較有效率的方法! 逐一人手命名檔案不合理呢!

說實話，如果我自已遇到這個問題不懂思考如何解決，我自已都過不了自己那一關!

現在假設我們要把檔案名稱含有"a_"的名字改為"abc_"

shek@shek-pc:~/folder$ ls -la
total 8
drwxrwxr-x   2 shek shek 4096 Oct  1 01:53 .
drwxr-xr-x 103 shek shek 4096 Oct  1 01:30 ..
-rw-rw-r--   1 shek shek    0 Oct  1 01:30 a_b.txt
-rw-rw-r--   1 shek shek    0 Oct  1 01:30 a_c.txt
-rw-rw-r--   1 shek shek    0 Oct  1 01:53 b.txt

一般來說命名多個檔案的linux command 會類似這樣:

for FILENAME in *; do mv $FILENAME Unix_$FILENAME; done

但我自已就想到用ls + awk + sed 解決這個問題，awk 和sed 非常實用，小弟真的要花點時間再深入看看awk 和 sed 有什麼其他功能...首先是要Filter我們想要的File名

shek@shek-pc:~/folder$ ls -la | grep a | awk '{print $9}'

a_b.txt
a_c.txt

之後便行for loop了，為了安全起見還是不直接執行，列印了想行的command再執行

shek@shek-pc:~/folder$ for i in $(ls -la | grep a | awk '{print $9}'); do echo mv $i $(echo $i|sed 's/a_/abc_/g') \&\&; done
mv a_b.txt abc_b.txt &&
mv a_c.txt abc_c.txt &&

之後執行以下command 便完成了!

mv a_b.txt abc_b.txt &&
mv a_c.txt abc_c.txt

shek@shek-pc:~/folder$ ls -la
total 8
drwxrwxr-x   2 shek shek 4096 Oct  1 02:02 .
drwxr-xr-x 103 shek shek 4096 Oct  1 01:30 ..
-rw-rw-r--   1 shek shek    0 Oct  1 01:30 abc_b.txt
-rw-rw-r--   1 shek shek    0 Oct  1 01:30 abc_c.txt
-rw-rw-r--   1 shek shek    0 Oct  1 01:53 b.txt