Skip to content

general-concepts.md

Latest commit

 

History

History
53 lines (37 loc) · 1.79 KB

general-concepts.md

File metadata and controls

53 lines (37 loc) · 1.79 KB
description General Concepts of Sequence Models

📚 General Concepts

👩‍🏫 Notation

In the context of text processing (e.g: Natural Language Processing NLP)

Symbol Description
$$X^{}$$ The tth word in the input sequence
$$Y^{}$$ The tth word in the output sequence
$$X^{(i)}$$ The tth word in the ith input sequence
$$Y^{(i)}$$ The tth word in the ith output sequence
$$T^{(i)}_x$$ The length of the ith input sequence
$$T^{(i)}_y$$ The length of the ith output sequence

🚀 One Hot Encoding

A way to represent words so we can treat with them easily

🔎 Example

Let's say that we have a dictionary that consists of 10 words (🤭) and the words of the dictionary are:

  • Car, Pen, Girl, Berry, Apple, Likes, The, And, Boy, Book.

Our $$X^{(i)}$$ is: The Girl Likes Apple And Berry

So we can represent this sequence like the following 👀

Car   -0)  ⌈ 0 ⌉   ⌈ 0 ⌉   ⌈ 0 ⌉   ⌈ 0 ⌉  ⌈ 0 ⌉   ⌈ 0 ⌉ 
Pen   -1)  | 0 |  | 0 |  | 0 |  | 0 |  | 0 |  | 0 |
Girl  -2)  | 0 |  | 1 |  | 0 |  | 0 |  | 0 |  | 0 |
Berry -3)  | 0 |  | 0 |  | 0 |  | 0 |  | 0 |  | 1 |
Apple -4)  | 0 |  | 0 |  | 0 |  | 1 |  | 0 |  | 0 |
Likes -5)  | 0 |  | 0 |  | 1 |  | 0 |  | 0 |  | 0 |
The   -6)  | 1 |  | 0 |  | 0 |  | 0 |  | 0 |  | 0 |
And   -7)  | 0 |  | 0 |  | 0 |  | 0 |  | 1 |  | 0 |
Boy   -8)  | 0 |  | 0 |  | 0 |  | 0 |  | 0 |  | 0 |
Book  -9)  ⌊ 0 ⌋   ⌊ 0 ⌋   ⌊ 0 ⌋   ⌊ 0 ⌋  ⌊ 0 ⌋   ⌊ 0 ⌋

By representing sequences in this way we can feed out data to neural networks ✨

🙄 Disadvantage

  • If our dictionary consists of 10,000 words so each vector will be 10,000 dimensional 🤕
  • This representation can not capture semantic features 💔