Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: aiDAPTIV Operator
description: Learn how to install the aiDAPTIV Cache Operator for Phison aiDAPTIVCache device support in Kubernetes
title: aiDAPTIVCache Operator
description: Learn how to install the aiDAPTIVCache Operator for Phison aiDAPTIVCache device support in Kubernetes
---

import { Steps, Aside } from '@astrojs/starlight/components';

This guide demonstrates how to install the **aiDAPTIV Cache Operator**, which is required for Kubernetes to recognize and utilize Phison aiDAPTIVCache devices (`phison.com/ai100`).
This guide demonstrates how to install the **aiDAPTIVCache Operator**, which is required for Kubernetes to recognize and utilize Phison aiDAPTIVCache devices (`phison.com/ai100`).

## Prerequisites

Expand All @@ -16,7 +16,7 @@ Before installing the operator, ensure you have:
- Sufficient cluster resources for the operator deployment

<Aside type="note">
The aiDAPTIV Cache Operator must be installed **before** deploying any workloads that require Phison aiDAPTIVCache devices. Without this operator, Kubernetes will not be able to recognize or allocate aiDAPTIVCache resources.
The aiDAPTIVCache Operator must be installed **before** deploying any workloads that require Phison aiDAPTIVCache devices. Without this operator, Kubernetes will not be able to recognize or allocate aiDAPTIVCache resources.
</Aside>

## Installation Steps
Expand All @@ -29,7 +29,7 @@ The aiDAPTIV Cache Operator must be installed **before** deploying any workloads
- Go to **Application** → **Store**
- This opens the Helm chart repository

2. **Add the aiDAPTIV Operator Chart**
2. **Add the aiDAPTIVCache Operator Chart**

Click **Import** button and provide the following information:

Expand Down Expand Up @@ -57,7 +57,7 @@ The aiDAPTIV Cache Operator must be installed **before** deploying any workloads

## What the Operator Does

The aiDAPTIV Cache Operator performs the following functions:
The aiDAPTIVCache Operator performs the following functions:

- **Resource Manager**: Manage aiDAPTIVCache devices
- **Device Discovery**: Automatically detects Phison aiDAPTIVCache devices on cluster nodes
Expand Down Expand Up @@ -86,10 +86,10 @@ If the operator pod fails to start:

## Next Steps

After successfully installing the aiDAPTIV Cache Operator, you can:
After successfully installing the aiDAPTIVCache Operator, you can:

- Deploy fine-tuning jobs that utilize aiDAPTIVCache devices (see [Deploy aiDAPTIV Finetune](/demos/08-aidaptiv-finetune))
- Deploy inference services with aiDAPTIVCache acceleration (see [Deploy aiDAPTIV Inference](/demos/09-aidaptiv-inference))
- Deploy fine-tuning jobs that utilize aiDAPTIVCache devices (see [Deploy aiDAPTIVCache Finetune](/demos/08-aidaptivcache-finetune))
- Deploy inference services with aiDAPTIVCache acceleration (see [Deploy aiDAPTIVCache Inference](/demos/09-aidaptivcache-inference))

<Aside type="tip">
When requesting aiDAPTIVCache devices in your workloads, use the resource key `phison.com/ai100` in both `requests` and `limits` sections of your pod specifications.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,30 +1,29 @@
---
title: aiDAPTIV Finetune
description: Fine-tune LLM models using aiDAPTIV Cache in OtterScale cluster.
title: aiDAPTIVCache Finetune
description: Fine-tune LLM models using aiDAPTIVCache in OtterScale cluster.
---

import { Steps, Aside, LinkCard } from '@astrojs/starlight/components';

This guide demonstrates how to fine-tune Large Language Models (LLMs) using aiDAPTIV Cache within the OtterScale cluster.
This guide demonstrates how to fine-tune Large Language Models (LLMs) using aiDAPTIVCache within the OtterScale cluster.

## Overview

aiDAPTIV Finetune provides efficient model fine-tuning capabilities with support for LoRA, full parameter training, and other training modes. Key features:
aiDAPTIVCache finetune provides efficient model fine-tuning capabilities with support for LoRA, full parameter training, and other training modes. Key features:

- Distributed training support (Multi-GPU)
- NFS storage integration for model input/output
- LoRA fine-tuning support
- Customizable training datasets
- Flexible resource allocation (vGPU, memory)

## Prerequisites

<Aside type="caution">
**Required**: Before using aiDAPTIV Finetune, you must install the **aiDAPTIV Cache Operator**.
**Required**: Before using aiDAPTIVCache finetune, you must install the **aiDAPTIVCache Operator**.

The operator enables Kubernetes to recognize and allocate Phison aiDAPTIVCache devices (`phison.com/ai100`) used by aiDAPTIV workloads.

See [aiDAPTIV Operator](/demo/07-aidaptiv-operator) for installation instructions.
See [aiDAPTIVCache Operator](/demos/07-aidaptivcache-operator) for installation instructions.
</Aside>

## Prepare Model and Data
Expand All @@ -40,7 +39,7 @@ This method uses OtterScale's NFS File System to store and access model files.
1. **Create NFS File System in OtterScale**:

Navigate to the Storage section:
- Go to `https://<your-domain>/scope/<scope-name>/service/file-system`
- Go to **Storage** → **File System**
- Create a new NFS File System or use an existing one
- Record the NFS server address (format: `10.102.197.0:/volumes/_nogroup/xxx`)

Expand Down Expand Up @@ -192,7 +191,7 @@ job:
name: finetune-job
backoffLimit: 1
restartPolicy: Never
ttlSecondsAfterFinished: 3600
ttlSecondsAfterFinished: 60
```

- `name`: Kubernetes Job name
Expand Down Expand Up @@ -415,7 +414,7 @@ job:
name: finetune-llama-job
backoffLimit: 1
restartPolicy: Never
ttlSecondsAfterFinished: 3600
ttlSecondsAfterFinished: 60

securityContext:
privileged: true
Expand Down Expand Up @@ -509,11 +508,8 @@ resources:
1. **Check Job Status**:

Navigate to the Jobs page:
```
https://<your-domain>/scope/<scope-name>/service/job
```

Find your fine-tune job under **Application → Job** and check its status.
- Go to **Applications** → **Jobs**
- Find your fine-tune job under your namespace and check its status.

2. **Retrieve Output Model**:

Expand All @@ -531,9 +527,9 @@ resources:
## Related Resources

<LinkCard
title="AiDaptiv Inference"
title="aiDaptivCache Inference"
description="Deploy fine-tuned models for inference"
href="/demos/09-aidaptiv-inference"
href="/demos/09-aidaptivcache-inference"
/>

<LinkCard
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: aiDAPTIV Inference
title: aiDAPTIVCache Inference
description: Deploy high-performance LLM inference service using aiDAPTIV Cache.
---

Expand All @@ -9,24 +9,22 @@ This guide demonstrates how to deploy a high-performance LLM inference service u

## Overview

aiDAPTIV Inference provides high-performance LLM inference service based on vLLM, supporting:
aiDAPTIVCache Inference provides high-performance LLM inference service based on vLLM, supporting:

- High-throughput serving
- Tensor parallelism (Multi-GPU)
- PagedAttention memory optimization
- KV Cache offloading (DRAM/SSD)
- KV Cache offloading
- Dynamic LoRA loading
- Prefix Caching
- OpenAI-compatible API

## Prerequisites

<Aside type="caution">
**Required**: Before using aiDAPTIV Inference, you must install the **aiDAPTIV Cache Operator**.
**Required**: Before using aiDAPTIVCache Inference, you must install the **aiDAPTIVCache Operator**.

The operator enables Kubernetes to recognize and allocate Phison aiDAPTIVCache devices (`phison.com/ai100`) used by aiDAPTIV workloads.

See [aiDAPTIV Operator](/demo/07-aidaptiv-operator) for installation instructions.
See [aiDAPTIVCache Operator](/demos/07-aidaptivcache-operator) for installation instructions.
</Aside>

## Prepare Model
Expand All @@ -42,7 +40,7 @@ This method uses OtterScale's NFS File System to store and access model files.
1. **Create NFS File System in OtterScale**:

Navigate to the Storage section:
- Go to `https://<your-domain>/scope/<scope-name>/service/file-system`
- Go to **Storage** → **File System**
- Create a new NFS File System or use an existing one
- Record the NFS server address (format: `10.102.197.0:/volumes/_nogroup/xxx`)

Expand Down Expand Up @@ -467,12 +465,8 @@ resources:

1. **Get Service Address**:

Navigate to the Services page:
```
https://<your-domain>/scope/<scope-name>/service/services
```

Find your inference service and note the NodePort.
- Go to **Application** → **Services**
- Find your inference service and note the NodePort.

2. **Test API Connection**:

Expand Down Expand Up @@ -546,9 +540,8 @@ resources:
1. **Check Deployment Status**:

Navigate to the Workloads page:
```
https://<your-domain>/scope/<scope-name>/service/workload
```
- Go to **Application** → **Workloads**
- Find your deployment

2. **View Pod Logs**:

Expand Down Expand Up @@ -601,9 +594,9 @@ resources:
## Related Resources

<LinkCard
title="aiDAPTIV Finetune"
title="aiDAPTIVCache Finetune"
description="Train and fine-tune LLM models"
href="/demos/08-aidaptiv-finetune"
href="/demos/08-aidaptivcache-finetune"
/>

<LinkCard
Expand Down
Loading