Skip to content

Commit 0970896

Browse files
authored
recipes_source/recipes/reasoning_about_shapes.py ๋ฒˆ์—ญ (#779)
* recipes_source/recipes/reasoning_about_shapes.py ๋ฒˆ์—ญ
1 parent 78d7662 commit 0970896

2 files changed

Lines changed: 17 additions & 20 deletions

File tree

โ€Žrecipes_source/recipes/reasoning_about_shapes.pyโ€Ž

Lines changed: 15 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,15 @@
11
"""
2-
Reasoning about Shapes in PyTorch
2+
PyTorch์˜ Shape๋“ค์— ๋Œ€ํ•œ ์ถ”๋ก 
33
=================================
4+
๋ฒˆ์—ญ: `์ด์˜์„ญ <https://github.com/0seob>`_
45
5-
When writing models with PyTorch, it is commonly the case that the parameters
6-
to a given layer depend on the shape of the output of the previous layer. For
7-
example, the ``in_features`` of an ``nn.Linear`` layer must match the
8-
``size(-1)`` of the input. For some layers, the shape computation involves
9-
complex equations, for example convolution operations.
6+
์ผ๋ฐ˜์ ์œผ๋กœ PyTorch๋กœ ๋ชจ๋ธ์„ ์ž‘์„ฑํ•  ๋•Œ ํŠน์ • ๊ณ„์ธต์˜ ๋งค๊ฐœ๋ณ€์ˆ˜๋Š” ์ด์ „ ๊ณ„์ธต์˜ ์ถœ๋ ฅ shape์— ๋”ฐ๋ผ ๋‹ฌ๋ผ์ง‘๋‹ˆ๋‹ค.
7+
์˜ˆ๋ฅผ ๋“ค์–ด, ``nn.Linear`` ๊ณ„์ธต์˜ ``in_features`` ๋Š” ์ž…๋ ฅ์˜ ``size(-1)`` ์™€ ์ผ์น˜ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
8+
๋ช‡๋ช‡ ๊ณ„์ธต์˜ ๊ฒฝ์šฐ, shape ๊ณ„์‚ฐ์€ ํ•ฉ์„ฑ๊ณฑ ์—ฐ์‚ฐ๊ณผ ๊ฐ™์€ ๋ณต์žกํ•œ ๋ฐฉ์ •์‹์„ ํฌํ•จํ•ฉ๋‹ˆ๋‹ค.
109
11-
One way around this is to run the forward pass with random inputs, but this is
12-
wasteful in terms of memory and compute.
10+
์ด๋ฅผ ๋žœ๋คํ•œ ์ž…๋ ฅ์œผ๋กœ ์ˆœ์ „ํŒŒ(forward pass)๋ฅผ ์‹คํ–‰ํ•˜์—ฌ ํ•ด๊ฒฐํ•  ์ˆ˜ ์žˆ์ง€๋งŒ, ์ด๋Š” ๋ฉ”๋ชจ๋ฆฌ์™€ ์ปดํ“จํŒ… ํŒŒ์›Œ๋ฅผ ๋‚ญ๋น„ํ•ฉ๋‹ˆ๋‹ค.
1311
14-
Instead, we can make use of the ``meta`` device to determine the output shapes
15-
of a layer without materializing any data.
12+
๋Œ€์‹ ์— ``meta`` ๋””๋ฐ”์ด์Šค๋ฅผ ํ™œ์šฉํ•œ๋‹ค๋ฉด ๋ฐ์ดํ„ฐ๋ฅผ ๊ตฌ์ฒดํ™”ํ•˜์ง€ ์•Š๊ณ ๋„ ๊ณ„์ธต์˜ ์ถœ๋ ฅ shape์„ ๊ฒฐ์ •ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
1613
"""
1714

1815
import torch
@@ -29,8 +26,8 @@
2926

3027

3128
##########################################################################
32-
# Observe that since data is not materialized, passing arbitrarily large
33-
# inputs will not significantly alter the time taken for shape computation.
29+
# ๋ฐ์ดํ„ฐ๊ฐ€ ๊ตฌ์ฒดํ™”๋˜์ง€ ์•Š๊ธฐ ๋•Œ๋ฌธ์— ์ž„์˜๋กœ ํฐ ์ž…๋ ฅ์„ ์ „๋‹ฌํ•ด๋„ shape ๊ณ„์‚ฐ์— ์†Œ์š”๋˜๋Š” ์‹œ๊ฐ„์ด
30+
# ํฌ๊ฒŒ ๋ณ€๊ฒฝ๋˜์ง€๋Š” ์•Š์Šต๋‹ˆ๋‹ค.
3431

3532
t_large = torch.rand(2**10, 3, 2**16, 2**16, device="meta")
3633
start = timeit.default_timer()
@@ -42,7 +39,7 @@
4239

4340

4441
######################################################
45-
# Consider an arbitrary network such as the following:
42+
# ๋‹ค์Œ๊ณผ ๊ฐ™์€ ์ž„์˜์˜ ๋„คํŠธ์›Œํฌ๋ฅผ ๊ฐ€์ •ํ•ฉ๋‹ˆ๋‹ค:
4643

4744
import torch.nn as nn
4845
import torch.nn.functional as F
@@ -61,23 +58,23 @@ def __init__(self):
6158
def forward(self, x):
6259
x = self.pool(F.relu(self.conv1(x)))
6360
x = self.pool(F.relu(self.conv2(x)))
64-
x = torch.flatten(x, 1) # flatten all dimensions except batch
61+
x = torch.flatten(x, 1) # ๋ฐฐ์น˜๋ฅผ ์ œ์™ธํ•œ ๋ชจ๋“  ์ฐจ์›์„ ํ‰ํƒ„ํ™” ํ•ฉ๋‹ˆ๋‹ค.
6562
x = F.relu(self.fc1(x))
6663
x = F.relu(self.fc2(x))
6764
x = self.fc3(x)
6865
return x
6966

7067

7168
###############################################################################
72-
# We can view the intermediate shapes within an entire network by registering a
73-
# forward hook to each layer that prints the shape of the output.
69+
# ๊ฐ๊ฐ์˜ ๊ณ„์ธต์— ์ถœ๋ ฅ์˜ shape์„ ์ธ์‡„ํ•˜๋Š” forward hook์„ ๋“ฑ๋กํ•˜์—ฌ ๋„คํŠธ์›Œํฌ์˜
70+
# ์ค‘๊ฐ„ shape์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
7471

7572
def fw_hook(module, input, output):
7673
print(f"Shape of output to {module} is {output.shape}.")
7774

7875

79-
# Any tensor created within this torch.device context manager will be
80-
# on the meta device.
76+
# torch.device context manager(with ๊ตฌ๋ฌธ) ๋‚ด๋ถ€์—์„œ ์ƒ์„ฑ๋œ ๋ชจ๋“  tensor๋Š”
77+
# meta ๋””๋ฐ”์ด์Šค ๋‚ด๋ถ€์— ์กด์žฌํ•ฉ๋‹ˆ๋‹ค.
8178
with torch.device("meta"):
8279
net = Net()
8380
inp = torch.randn((1024, 3, 32, 32))

โ€Žrecipes_source/recipes_index.rstโ€Ž

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -122,8 +122,8 @@ Recipes are bite-sized bite-sized, actionable examples of how to use specific Py
122122
:tags: Basics
123123

124124
.. customcarditem::
125-
:header: Reasoning about Shapes in PyTorch
126-
:card_description: Learn how to use the meta device to reason about shapes in your model.
125+
:header: PyTorch์˜ Shape์— ๋Œ€ํ•œ ์ถ”๋ก 
126+
:card_description: meta ๋””๋ฐ”์ด์Šค๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ชจ๋ธ์˜ shape์„ ์ถ”๋ก ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์•Œ์•„๋ด…๋‹ˆ๋‹ค.
127127
:image: ../_static/img/thumbnails/cropped/generic-pytorch-logo.png
128128
:link: ../recipes/recipes/reasoning_about_shapes.html
129129
:tags: Basics

0 commit comments

Comments
ย (0)