Skip to content

Data Transformation & Visualisation

davidvanr21 edited this page Nov 26, 2020 · 3 revisions

Resources:

RDW

Overheid: https://data.overheid.nl/datasets?facet_classification[0]=national_coverage&sort=sys_modified desc&facet_theme[0]=http%3A//standaarden.overheid.nl/owms/terms/Verkeer_(thema)

CBS: https://opendata.cbs.nl/statline/#/CBS/en/navigatieScherm/thema

https://docs.google.com/spreadsheets/d/1NtJCWFFtU9uT7gTxwtwLk92W4K0QROnn3d8jaVOUew0/edit#gid=0

API bij ANWB checken voor drukte op wegen > files

Data van CBS inladen: https://vizhub.com/Razpudding/5fd132d0c65d4df3b92a116833f1259e?edit=files&file=index.js

Used Data

This is the link to the data I used from the RDW open dataset. From this data I will use 2 variables:

  • Catalogusprijs (list price)
  • Datum eerste

To get some idea of the data I'm using, I used the visualiser inside the RDW website. First I was looking inside the list prices and saw directly that around 7 million cars have a undefined value. This is good to know because I will see a lot undefined values when I will start coding.

Data Transformation

Some test code with beginning of good code

console.log('Script loaded');

// // Loading data with API
// // Defining API from RDW
// const dataUrl = "https://opendata.rdw.nl/resource/m9d7-ebf2.json";

// // Fetching the data and transform it to JSON
// // source: https://www.youtube.com/watch?v=uxf0--uiX0I&ab_channel=TheCodingTrain
// async function getData() {
//     const response = await fetch(dataUrl);
//     const data = await response.json();
//     console.log(data);
// }

// getData();

// // const merk = dataUrl.merk;

// Variabele naam veranderen
const rdwData = dataJS;
console.log(rdwData);

// Testen om waarde uit 1 array te halen
function empty() {
    if (rdwData[0].merk === 'TOYOTA') {
        return 'it works';
    }
}

console.log(empty());

// Loopen om data uit array te halen
var catalogusprijs = for (answer in rdwData){
    console.log(rdwData[answer].catalogusprijs)
};

// Catalogus prijs moet een var worden
// Var omzetten van string naar number
// Lege strings weghalen
// Alles wat tussen 0 en 50, 50 en 100, 100 en 200 of 200+ is filteren

I wrote what I wanted to happen in the code:

// Console print check
console.log('Script loaded');
console.log(rdwData);

// 1.
// Map for each array to get the value of "catalogusprijs"
// or
// Filter the "catalogusprijs" value of all the objects

// 2.
// Lege strings weghalen

// 3.
// Change "catalogusprijs" from string to number

// 4.
// Alles wat tussen 0 en 50, 50 en 100, 100 en 200 of 200+ is filteren 
// If (>0 & <50000){}

For filtering out the "catalogusprijs" I looked at my exercise from funFunFunction to see what are the possibilitys. The filter function looks good for this I think. This is the example I used for that exercise.

var dogs = animals.filter(function(animal){
    return animal.species === 'dog'
})
var catalogusprijs = rdwData.filter(function(price){
    return price.catalogusprijs === '23644'
})

console.log(catalogusprijs);

This works. But is not ideal for my code.

This was a example, lets test this method

var isDog = function(animal){
    return animal.species === 'dog'
}

var dogs = animals.filter(isDog)

I dont think I'm using the write code because I do not have to filter the data. I need the seperate the data

I've tryed many different kind of code

let prijs = []
for (answer of rdwData){
    console.log(answer.catalogusprijs)
}

With this code, all my data is indefined

// Filter the "catalogusprijs" value of all the objects
let kolomNaam = "catalogusprijs"
let answers = []
for (answer of rdwData){
  answers.push(answer.kolomNaam)
}

console.log(answers);

Dynamic search you use bracket signs

answers.push(answer[kolomNaam])

Chrome inspector trick!

  • Right mouse click in console
  • Store as global variable
  • console.table(temp1)

A great table will be shown

map = a function for each element in an array

// Filter the "catalogusprijs" value of all the objects
let kolomNaam = "catalogusprijs"
let answers = []

for (answer of rdwData){
  answers.push(answer[kolomNaam])
}

console.log(answers);
let kolomNaam = "catalogusprijs"
let answers = firstFunction(answers, kolomNaam)

function firstFunction(answers){
    if(answers.length > 0){
      for (answer of rdwData){
      answers.push(answer[kolomNaam])
      }
    }
}

console.log(answers);
var catalogusprijsNum = parseFloat(catalogusprijs)

console.log(catalogusprijs);

var catalogusprijs = rdwData.filter(function(price){
    return price.catalogusprijs === '23644'
})

console.log(catalogusprijs);

Filter the prices
var catalogusprijsSelect = function(price){
    return 
}
var catalogusprijs = rdwData.filter(catalogusprijsSelect)

console.log(catalogusprijs);

var catalogusprijsSelect = function(price){
  return 
}
var catalogusprijs = rdwData.filter(catalogusprijsSelect)

console.log(catalogusprijs);
Clone this wiki locally