|
1 | 1 | """ |
2 | | -Model Freezing in TorchScript |
| 2 | +TorchScriptλ‘ λͺ¨λΈ λκ²°νκΈ° |
3 | 3 | ============================= |
| 4 | +λ²μ : `κΉμ§νΈ <https://github.com/jiho3004/>`_ |
4 | 5 |
|
5 | | -In this tutorial, we introduce the syntax for *model freezing* in TorchScript. |
6 | | -Freezing is the process of inlining Pytorch module parameters and attributes |
7 | | -values into the TorchScript internal representation. Parameter and attribute |
8 | | -values are treated as final values and they cannot be modified in the resulting |
9 | | -Frozen module. |
| 6 | +μ΄ νν 리μΌμμλ, TorchScriptλ‘ *λͺ¨λΈ λκ²°* νλ λ¬Έλ²μ μκ°ν©λλ€. |
| 7 | +λκ²°μ νμ΄ν μΉ λͺ¨λμ λ§€κ°λ³μμ μμ± κ°λ€μ TorchScript λ΄λΆ ννμΌλ‘ μΈλΌμ΄λ(inlining)νλ κ³Όμ μ
λλ€. |
| 8 | +λ§€κ°λ³μμ μμ± κ°λ€μ μ΅μ’
κ°μΌλ‘ μ²λ¦¬λλ©° λκ²°λ λͺ¨λμμ μμ λ μ μμ΅λλ€. |
10 | 9 |
|
11 | | -Basic Syntax |
| 10 | +κΈ°λ³Έ λ¬Έλ² |
12 | 11 | ------------ |
13 | | -Model freezing can be invoked using API below: |
| 12 | +
|
| 13 | +λͺ¨λΈ λκ²°μ μλ APIλ₯Ό μ¬μ©νμ¬ νΈμΆν μ μμ΅λλ€: |
14 | 14 |
|
15 | 15 | ``torch.jit.freeze(mod : ScriptModule, names : str[]) -> SciptModule`` |
16 | 16 |
|
17 | | -Note the input module can either be the result of scripting or tracing. |
18 | | -See https://tutorials.pytorch.kr/beginner/Intro_to_TorchScript_tutorial.html |
| 17 | +μ
λ ₯ λͺ¨λμ μ€ν¬λ¦½ν
(scripting) νΉμ μΆμ (tracing)μ μ¬μ©ν κ²°κ³Όμ
λλ€. |
| 18 | +`TorchScript μκ° νν λ¦¬μΌ <https://tutorials.pytorch.kr/beginner/Intro_to_TorchScript_tutorial.html>`_ |
| 19 | +μ μ°Έμ‘°νμΈμ. |
19 | 20 |
|
20 | | -Next, we demonstrate how freezing works using an example: |
| 21 | +λ€μμΌλ‘, μμ λ₯Ό ν΅ν΄ λκ²°μ΄ μ΄λ€ λ°©μμΌλ‘ λμνλμ§ νμΈν©λλ€: |
21 | 22 | """ |
22 | 23 |
|
23 | 24 | import torch, time |
@@ -58,17 +59,15 @@ def version(self): |
58 | 59 |
|
59 | 60 | try: |
60 | 61 | print(fnet.conv1.bias) |
61 | | - # without exception handling, prints: |
| 62 | + # μμΈ μ²λ¦¬κ° μμ μ 'conv1' μ΄λΌλ μ΄λ¦κ³Ό ν¨κ» λ€μμ μΆλ ₯ν©λλ€. |
62 | 63 | # RuntimeError: __torch__.z.___torch_mangle_3.Net does not have a field |
63 | | - # with name 'conv1' |
64 | 64 | except RuntimeError: |
65 | 65 | print("field 'conv1' is inlined. It does not exist in 'fnet'") |
66 | 66 |
|
67 | 67 | try: |
68 | 68 | fnet.version() |
69 | | - # without exception handling, prints: |
| 69 | + # μμΈ μ²λ¦¬κ° μμ μ 'version' μ΄λΌλ μ΄λ¦κ³Ό ν¨κ» λ€μμ μΆλ ₯ν©λλ€. |
70 | 70 | # RuntimeError: __torch__.z.___torch_mangle_3.Net does not have a field |
71 | | - # with name 'version' |
72 | 71 | except RuntimeError: |
73 | 72 | print("method 'version' is not deleted in fnet. Only 'forward' is preserved") |
74 | 73 |
|
@@ -108,27 +107,25 @@ def version(self): |
108 | 107 | print("Frozen - Inference time: {0:5.2f}".format(end-start), flush =True) |
109 | 108 |
|
110 | 109 | ############################################################### |
111 | | -# On my machine, I measured the time: |
| 110 | +# κ°μΈ λ¨Έμ μμ μκ°μ μΈ‘μ ν κ²°κ³Όμ
λλ€: |
112 | 111 | # |
113 | 112 | # * Scripted - Warm up time: 0.0107 |
114 | 113 | # * Frozen - Warm up time: 0.0048 |
115 | 114 | # * Scripted - Inference: 1.35 |
116 | 115 | # * Frozen - Inference time: 1.17 |
117 | 116 |
|
118 | 117 | ############################################################### |
119 | | -# In our example, warm up time measures the first two runs. The frozen model |
120 | | -# is 50% faster than the scripted model. On some more complex models, we |
121 | | -# observed even higher speed up of warm up time. freezing achieves this speed up |
122 | | -# because it is doing some the work TorchScript has to do when the first couple |
123 | | -# runs are initiated. |
| 118 | +# μ΄ μμ μμ, μλ°μ
μκ°μ μ΅μ΄ λ λ² μ€νν λ μΈ‘μ ν©λλ€. |
| 119 | +# λκ²°λ λͺ¨λΈμ΄ μ€ν¬λ¦½νΈλ λͺ¨λΈλ³΄λ€ 50% λ λΉ λ¦
λλ€. |
| 120 | +# λ³΄λ€ λ³΅μ‘ν λͺ¨λΈμμλ μλ°μ
μκ°μ΄ λμ± λΉ¨λΌμ§λλ€. |
| 121 | +# μ΅μ΄ λ λ²μ μ€νμ μ΄κΈ°νν λ TorchScriptκ° ν΄μΌ ν μΌμ μΌλΆλ₯Ό λκ²°μ΄ νκ³ μκΈ° λλ¬Έμ μλ κ°μ μ΄ μΌμ΄λ©λλ€. |
124 | 122 | # |
125 | | -# Inference time measures inference execution time after the model is warmed up. |
126 | | -# Although we observed significant variation in execution time, the |
127 | | -# frozen model is often about 15% faster than the scripted model. When input is larger, |
128 | | -# we observe a smaller speed up because the execution is dominated by tensor operations. |
| 123 | +# μΆλ‘ μκ°μ λͺ¨λΈμ΄ μλ°μ
λκ³ λ λ€, μΆλ‘ μ μ€ν μκ°μ μΈ‘μ ν©λλ€. |
| 124 | +# μ€ν μκ°μ λ§μ νΈμ°¨κ° μκΈ°λ νμ§λ§, λκ° λκ²°λ λͺ¨λΈμ΄ μ€ν¬λ¦½νΈλ λͺ¨λΈλ³΄λ€ μ½ 15% λ λΉ λ¦
λλ€. |
| 125 | +# μ€ν μκ°μ tensor μ°μ°μ μν΄ μ§λ°°λκΈ° λλ¬Έμ μ
λ ₯μ ν¬κΈ°κ° λ 컀μ§λ©΄ μλ κ°μ μ λλ λ μμμ§λλ€. |
129 | 126 |
|
130 | 127 | ############################################################### |
131 | | -# Conclusion |
| 128 | +# κ²°λ‘ |
132 | 129 | # ----------- |
133 | | -# In this tutorial, we learned about model freezing. Freezing is a useful technique to |
134 | | -# optimize models for inference and it also can significantly reduce TorchScript warmup time. |
| 130 | +# μ΄ νν 리μΌμμλ λͺ¨λΈ λκ²°μ λν΄ λ°°μ μ΅λλ€. |
| 131 | +# λκ²°μ μΆλ‘ μ λͺ¨λΈ μ΅μ νλ₯Ό ν μ μλ μ μ©ν κΈ°λ²μ΄λ©° TorchScript μλ°μ
μκ°μ ν¬κ² μ€μ
λλ€. |
0 commit comments