- There is an old idea from Kendo which seems to find its way to the new world of cutting-edge technology. The idea is that you learn a martial art in four stages: big, strong, fast, light. ‘Big’ is the phase where all movements have to be big and correct. One here focuses on correct techniques, and one’s muscles adapt to the new movements. While doing big movements, they unconsciously start becoming strong. ‘Strong’ is the next phase, when one focuses on strong movements. We have learned how to do it correctly, and now we add strength, and subconsciously they become faster and faster. While learning ‘Fast’, we start ‘cutting corners’, and adopt a certain ‘parsimony’. This parsimony builds ‘Light’, which means ‘just enough’. In this phase, the practitioner is a master, who does everything correctly, and movements can shift from strong to fast and back to strong, and yet they seem effortless and light. This is the road to mastery of the given martial art, and to an art in general.
- A machine-learning model transforms its input data into meaningful outputs, a process that is “learned” from exposure to known examples of inputs and outputs. Therefore, the central problem in machine learning and deep learning is to meaningfully transform data: in other words, to learn useful representations of the input data at hand—representations that get us closer to the expected output. Before we go any further: what’s a representation? At its core, it’s a different way to look at data—to represent or encode data. For instance, a color image can be encoded in the RGB format (red-green-blue) or in the HSV format (hue-saturation-value): these are two different representations of the same data. Some tasks that may be difficult with one representation can become easy with another. For example, the task “select all red pixels in the image” is simpler in the RG format, whereas “make the image less saturated” is simpler in the HSV format. Machine-learning models are all about finding appropriate representations for their input data—transformations of the data that make it more amenable to the task at hand, such as a classification task.
- Machine learning is, technically: searching for useful representations of some input data, within a predefined space of possibilities (hypothesis space), using guidance from a feedback signal.
#higlights #quotes #machinelearning