Log loss

 

Introduction

Log Loss is the most important classification metric based on probabilities.

It's hard to interpret raw log-loss values, but log-loss is still a good metric for comparing models. For any given problem, a lower log-loss value means better predictions.

Log Loss is a slight twist on something called the Likelihood Function. In fact, Log Loss is -1 * the log of the likelihood function. So, we will start by understanding the likelihood function.

The likelihood function answers the question "How likely did the model think the actually observed set of outcomes was." If that sounds confusing, an example should help.

Example

A model predicts probabilities of [0.8, 0.4, 0.1] for three houses. The first two houses were sold, and the last one was not sold. So the actual outcomes could be represented numeically as [1, 1, 0].


Let's step through these predictions one at a time to iteratively calculate the likelihood function.

The first house sold, and the model said that was 80% likely. So, the likelihood function after looking at one prediction is 0.8.

The second house sold, and the model said that was 40% likely. There is a rule of probability that the probability of multiple independent events is the product of their individual probabilities. So, we get the combined likelihood from the first two predictions by multiplying their associated probabilities. That is 0.8 * 0.4, which happens to be 0.32.

Now we get to our third prediction. That home did not sell. The model said it was 10% likely to sell. That means it was 90% likely to not sell. So, the observed outcome of not selling was 90% likely according to the model. So, we multiply the previous result of 0.32 by 0.9.

We could step through all of our predictions. Each time we'd find the probability associated with the outcome that actually occurred, and we'd multiply that by the previous result. That's the likelihood.


From Likelihood to Log Loss

Each prediction is between 0 and 1. If you multiply enough numbers in this range, the result gets so small that computers can't keep track of it. So, as a clever computational trick, we instead keep track of the log of the Likelihood. This is in a range that's easy to keep track of. We multiply this by negative 1 to maintain a common convention that lower loss scores are better.

Comments

Post a Comment

Popular posts from this blog

Presentation_Rashmi

Image
 SLIDE 1 What is SVM? “Support Vector Machine” (SVM) is a supervised  machine learning algorithm  which can be used for both classification or regression challenges. However,  it is mostly used in classification problems. SLIDE 2 how does it work ? Identify the right hyper-plane (Scenario-1): SLIDE3: Identify the right hyper-plane (Scenario-2):  SLIDE4: Here, maximizing the distances between nearest data point (either class) and hyper-plane will help us to decide the right hyper-plane. This distance is called as  Margin SLIDE5: Identify the right hyper-plane (Scenario-3): SLIDE6: Find the hyper-plane to segregate to classes  SLIDE  Cost function and Gradient Function In the SVM algorithm, we are looking to maximize the margin between the data points and the hyperplane. The loss function that helps maximize the margin is hinge loss. SLIDE  Regularization The objective of the regularization parameter is to balance the margin maximizat...
Read more

MySQL : Structured Query Language

Image
The acronym SQL stands for Structured Query Language SQL is the standard language for data manipulation SQL is the standard language for relational database management. Three Main Uses of SQL are : Read ,Write , Update SQL is used for many things; however, Data Retrieval  was stressed as THE MAIN THING used by Data Scientist Data Scientist might do a little of the analysis using SQL, however the main thing they're using SQL for is data retrieval. Learning Objective: Explain why thinking before coding is important  Explain why it is important to understand how data in a database is relate to each other  Describe what a database is  Think before you do:    What is the problem you are trying to do..?     What I mean by this is it's easy when you're doing work or  we're trying to produce a result or solve a problem to just go in and start doing.  By that I mean you go in and start to write code or maybe start to write some  type of emai...
Read more

spoken

  Machine learning  is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.  basically Machine learning  focuses on the development of computer programs that can access data and use it learn for themselves. Currently,  machine learning  has been used in multiple fields and industries. For  example , medical diagnosis, image processing, prediction, classification,   regression etc Slide 2:  Machine Learning is categories in 3 parts  Supervised Learning  Unsupervised Learning Reinforecement Learning Slide 3: Supervised learning is where you have input variables (x) and an output variable (Y) and you use an algorithm to learn the mapping function from the input to the output. Y = f(X) The goal is to approximate the mapping function so well that when you have new input data (x) that you can predict the output variable...
Read more