Local LLM 적용

[jhgoh]

Local LLM을 설치해 사용해 보려고 함. naong에 quantized model을 올려 시험 해 볼 수 있을 것 같음.

[jhgoh]

설치 완료. qwen 올려둠.

jhgoh@naong:~$ !OL
OLLAMA_MODELS=/srv/ollama/models ollama list
NAME                       ID              SIZE      MODIFIED       
nomic-embed-text:latest    0a109f422b47    274 MB    18 minutes ago    
qwen3:14b                  bdbd181c33f2    9.3 GB    19 minutes ago    

jhgoh@naong:~$ nvidia-smi 
Fri Aug 28 13:58:51 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.173.02             Driver Version: 580.173.02     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1080 Ti     On  |   00000000:01:00.0 Off |                  N/A |
| 23%   45C    P2            215W /  220W |    9393MiB /  11264MiB |     99%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A            1510      C   ...local/lib/ollama/llama-server       9388MiB |
+-----------------------------------------------------------------------------------------+

[jhgoh]

jhgoh@lugia:~$ curl -s http://naong:11434/api/generate -d '{"model":"qwen3:14b","prompt":"안녕! 네 소개를 한 문장으로 해줘.","stream":false}'
{"model":"qwen3:14b","created_at":"2026-08-28T13:59:00.80804235Z","response":"안녕하세요! 저는 Qwen으로, 질문에 답변하고, 콘텐츠를 생성하며, 대화를 도와드릴 수 있는 인공지능 어시스턴트입니다. 다양한 언어를 지원하며, 정확하고 유용한 정보를 제공하기 위해 노력하고 있어요.","thinking":"Okay, the user asked me to introduce myself in one sentence. Let me think about how to do that.\n\nFirst, I need to cover my main functions. I can help with answering questions, creating content, and having conversations. That's a good start.\n\nI should mention the languages I support. The user might be interested in knowing that I can communicate in multiple languages, including Korean. That's important for accessibility.\n\nAlso, I should highlight my ability to provide accurate and helpful information. Users want reliable assistance. Maybe add something about being friendly and approachable to make the tone more welcoming.\n\nWait, the user wrote the query in Korean, so maybe they prefer the response in Korean. But the initial instruction was in Korean, but the answer needs to be in one sentence. Let me check the original request again.\n\nThe user wrote: \"안녕! 네 소개를 한 문장으로 해줘.\" Which means \"Hello! Please introduce yourself in one sentence.\" So they want the response in Korean. But the example response is in English. Hmm, maybe they want the answer in Korean. Let me adjust that.\n\nSo, in Korean, the response would be: \"안녕하세요! 저는 Qwen으로, 질문에 답변하고, 콘텐츠를 생성하며, 대화를 도와드릴 수 있는 인공지능 어시스턴트입니다. 다양한 언어를 지원하며, 정확하고 유용한 정보를 제공하기 위해 노력하고 있어요.\"\n\nWait, but the user might want the response in the same language as the query. Since the query was in Korean, the answer should be in Korean. But the initial example response was in English. Maybe the user is testing if I can switch languages. But the instruction was to introduce myself in one sentence, and the user's message is in Korean. So the answer should be in Korean. Let me make sure.\n\nAlso, I need to keep it concise, one sentence. Let me structure it properly. Start with a greeting, then state my role, mention language support, and my goal. That should cover it without being too long.\n","done":true,"done_reason":"stop","context":[151644,872,198,126246,144370,0,65553,97,142559,18411,61298,53435,40853,42039,60716,144131,13,608,26865,151645,198,151644,77091,198,151667,198,32313,11,279,1196,4588,752,311,19131,7037,304,825,11652,13,6771,752,1744,911,1246,311,653,429,382,5338,11,358,1184,311,3421,847,1887,5746,13,358,646,1492,448,35764,4755,11,6825,2213,11,323,3432,20959,13,2938,594,264,1661,1191,382,40,1265,6286,279,15459,358,1824,13,576,1196,2578,387,8014,304,14063,429,358,646,19032,304,5248,15459,11,2670,16134,13,2938,594,2989,369,39700,382,13394,11,358,1265,11167,847,5726,311,3410,13382,323,10950,1995,13,14627,1366,14720,12994,13,10696,912,2494,911,1660,11657,323,5486,480,311,1281,279,16232,803,35287,382,14190,11,279,1196,6139,279,3239,304,16134,11,773,7196,807,10702,279,2033,304,16134,13,1988,279,2856,7600,572,304,16134,11,714,279,4226,3880,311,387,304,825,11652,13,6771,752,1779,279,4024,1681,1549,382,785,1196,6139,25,330,126246,144370,0,65553,97,142559,18411,61298,53435,40853,42039,60716,144131,1189,15920,3363,330,9707,0,5209,19131,6133,304,825,11652,1189,2055,807,1366,279,2033,304,16134,13,1988,279,3110,2033,374,304,6364,13,88190,11,7196,807,1366,279,4226,304,16134,13,6771,752,7500,429,382,4416,11,304,16134,11,279,2033,1035,387,25,330,126246,144370,91145,0,134561,1207,16948,42039,11,138520,19391,143604,126204,11,3315,121,246,144137,142852,18411,51917,130705,11,60960,56290,18411,129392,80573,29346,135379,28733,64521,58677,78125,21329,66019,124685,134619,94152,28626,78952,13,135392,139957,31079,18411,132185,130705,11,36055,133085,126204,126310,26699,23573,60039,18411,130094,66425,130039,136111,126204,127353,35711,2217,14190,11,714,279,1196,2578,1366,279,2033,304,279,1852,4128,438,279,3239,13,8704,279,3239,572,304,16134,11,279,4226,1265,387,304,16134,13,1988,279,2856,3110,2033,572,304,6364,13,10696,279,1196,374,7497,421,358,646,3398,15459,13,1988,279,7600,572,311,19131,7037,304,825,11652,11,323,279,1196,594,1943,374,304,16134,13,2055,279,4226,1265,387,304,16134,13,6771,752,1281,2704,382,13394,11,358,1184,311,2506,432,63594,11,825,11652,13,6771,752,5944,432,10277,13,5145,448,264,42113,11,1221,1584,847,3476,11,6286,4128,1824,11,323,847,5795,13,2938,1265,3421,432,2041,1660,2238,1293,624,151668,271,126246,144370,91145,0,134561,1207,16948,42039,11,138520,19391,143604,126204,11,3315,121,246,144137,142852,18411,51917,130705,11,60960,56290,18411,129392,80573,29346,135379,28733,64521,58677,78125,21329,66019,124685,134619,94152,28626,78952,13,135392,139957,31079,18411,132185,130705,11,36055,133085,126204,126310,26699,23573,60039,18411,130094,66425,130039,136111,126204,127353,35711,13],"total_duration":18259485764,"load_duration":1413065,"prompt_eval_count":24,"prompt_eval_duration":105410000,"eval_count":501,"eval_duration":18018712000}```

[jhgoh]

[jhgoh]

[jhgoh]