본문 바로가기
728x90

분류 전체보기

 (430)
[PyTorch] Modify Gradient while Backward Propagation Using Hook Hooks in PyTorch allow you to execute custom functions at specific points during the forward or backward passes of your neural network. 1. Understanding the Gradient Flow In PyTorchWhen you call loss.backward(), the gradients are computed and stored in the grad attribute of each parameter.Forward Pass: You pass your input through the network to get the output.input tensor Xmodel parameter tensor..
[Math] Viewing Deep Learning From Maximum Likelihood Estimation Perspective We can see finding deep learning model's parameters from maximum likelihood estimation perspective.1. Normal distribution1.1 SettingLet's say we have a model μ = wx+band have a dataset {(x₁, y₁), ..., (x₁, y₅), (x₂, y₆), ... , (x₂, y₁₁), (x₃, y₁₂), ...} consists of n samples.1.2 Assume a probability distributionLet's assume the observed value y, given x, follows normal distribution with mean μ=w..
[Paper Review] Robust Speech Recognition via Large-Scale Weak Supervision Robust Speech Recognition via Large-Scale Weak SupervisionWe study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standardarxiv.org0. AbstractSuggest large-scale and weakly-supervised speech processing mode..
[Paper Review] Conformer: Convolution-augmented Transformer for Speech Recognition Conformer: Convolution-augmented Transformer for Speech RecognitionRecently Transformer and Convolution neural network (CNN) based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (RNNs). Transformer models are good at capturing content-based global interacarxiv.org0. AbstractTransformer models are good at capturing content-based ..
[Paper Review] Sequence Transduction with Recurrent Neural Networks Sequence Transduction with Recurrent Neural NetworksMany machine learning tasks can be expressed as the transformation---or \emph{transduction}---of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to name but a few. Onarxiv.org0. AbstractMany machine learning tasks can be expressed as the transformation—or ..
[Paper Review] Neural RRT*: Learning-Based Optimal Path Planning Neural RRT*: Learning-Based Optimal Path PlanningRapidly random-exploring tree (RRT) and its variants are very popular due to their ability to quickly and efficiently explore the state space. However, they suffer sensitivity to the initial solution and slow convergence to the optimal solution, which meanieeexplore.ieee.org0. AbstractRapidly random-exploring tree (RRT) is popular path planning al..
[Paper Review] Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks https://www.cs.toronto.edu/~graves/icml_2006.pdf0. AbstractIn speech recognition, for example, an acoustic signal is transcribed into words or sub-word units.RNNs are powerful sequence learners but there are two problems. 0.1 Pre-segmented training dataExample: Let's say we have an audio clip of someone saying "Hello world". Pre-segmented data might look like this:"He-" (0.0s - 0.2s) "-llo" (0.2..
[Paper Review] Attention Is All You Need Attention Is All You NeedThe dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a newarxiv.org0. AbstractThe dominant sequence transduction models are based on complex recurrent or convolutional neural n..
728x90

티스토리툴바