Deep Learning - 1.1 Data manipulation

Before diving into the deep learning, we have to learn basic skills which are sorting, manipulating and processing data.

• Create Tensor
• Operations
• Broadcasting
• Indexing, Slicing
• Saving Memory
• Conversion to other python objects

1. Tensor

We have to handle multi dimention array which is called "Tensor". All of data will be represented as tensor. It has two different features from numpy's ndarray.

- GPU is well-supported to accelerate tensor computation whereas NumPy only supports CPU computation.

- Tensor supports automatic differentiation

These two properties makes tensor more suitable to deep learning

 

2. Create tensor

create tensor

shape

total number of elements

reshape

You don't have to do the division (height, witdth) on your own. Just replacing the other size into -1, tensors automatically calculate the size.

you can also initialize all of the elements into specific values such as zero or one when creating a tensor.

Code above creates a tensor which is randomly sampled from a standard Gaussian normal distribution with a mean of 0 and a standard deviation of 1.

You can also specify the exact values of tensor.

 

3. Operations

Elementwise function apply a standard scalar operation to each element of an array. Two tensors have to be identical-shaped tensors of arbitrary shape.

Unary operators, for example exponentiation, can also be applied elementwise

concatenate

logical statements : compare each value at the same position of tensor(e.g. ==, >, <)

sum

 

4. Broadcasting

To perform elementwise operation with two tensors of different shape, you can expand a tensor to match up each other by invoking broadcasting mechanism. 

5. Indexing and Slicing

Aware that ranges are specified to include the first but before the last element

 

6. Saving memory

This means that python dereference the tensor that Y used to point to and instead point Y at the newly allocated memory

At deep learning we update huge tensors several times through operations per second, so This way is undesireable.

To reduce memory overhead we can use in-place operations like below.

If X is not reused you can use += also.

 

7. Conversion to Other Python Objects

Let's try converting tensor to python numpy, and vice versa.

Convert a size-1 tensor to a python scalar