# data[].attributes.body
Short-term rentals platforms such as Airbnb have made it easy for homeowners to list their apartments or spare rooms online.

Since Airbnb is a market, the amount a host can charge is ultimately tied to market prices and one challenge that Airbnb hosts face is determining the best price to rent out a space.

# data[].attributes.body
Machine Learning can be used anywhere from automating mundane tasks to offering intelligent insights.

Predictive analytics are particularly useful and can help Airbnb hosts price their apartments, given the apartments' traits vis-a-vis competing offerings in the market. Examples include:

training regression models to predict the target rent rates the hosts should charge, using a basket of predictors like the apartments' locations and nearness to amenities and attractions, the apartment sizes and furnishings

making use of classification models (Decision Tree, ANN, etc.) to explore whether the apartment attributes improve (Y/N) rent rates

combining the results of various models with Ensemble algorithms for "Wisdom of the Crowd" effects

# data[].attributes.body
The number of bedrooms available in the Airbnb listing for stay will also impact the price to rent out the space at. As such, we will perform data profiling on this column by conducting a semantic check. It is noted that there are 3 empty values found in this column. This would mean that the data for the AirBnB listing may be incomplete. As such, these empty values will be removed. Moreover, when we dive in to the min, max and average, there seems to be two extreme ends i.e 0 and 50 bedrooms which may indicate the presence of an outlier. A bar chart is hence used to analyze the data better and remove the outlier. Upon checking in detail, the outlier is a supposed accommodation for a family of 5 pax. However, it was incorrectly recorded as 50 bedrooms which can influence the pricing of the AirBnB stay depending on the number of pax that can be accommodated. Therefore, this outlier was removed to enhance the quality of the data.

# data[].attributes.body
In this column, we conduct another profiling for the room type to identify the uniqueness of the data. There are 24 unique count in this column however when we dive in further, there is an outlier which describes the room as Castle. This will impact how the data is used to analyse the relationship between the room type and the price of the AirBnb listing. In order to resolve this, we replaced the Castle to Others so as to narrow down the category for easier analysis.

# data[].attributes.headline
one DATAFRAME to rule them all ... data blending

# data[].attributes.subject
one DATAFRAME to rule them all ... data blending

# data[].attributes.body
To facilitate the use of the Airbnb data in the machine learning models (features engineering), data blending was performed to merge the 3 datasets from insideairbnb.com.

Airbnb listings

the two listings comprising hosts who registered in different time periods were appended using Data Union

both contained the same data headers (i.e., 26 columns of identical datapoints) which were automatically matched using Tibco Spotfire

the unified Airbnb listings had 7908 rows vertically stacked in a new data table

Airbnb neighbourhood

the data comprised the "Neighbourhood" column found in Airbnb listing + a categorical column "Region" with labels Central, North, East, West, etc.

Left outer join was used to add the "Region" column to the unified Airbnb listings

the joined Airbnb data table had 27 columns

# data[].attributes.body
We further transformed the data from tall and skinny to wide structure so that it is easier to illustrate the analysis of our findings. From this, we found that the regions with the highest average price per room for an Entire Home are most expensive in the Central followed by East, West, North and North East being the least expensive.

Private rooms are found to be most expensive in Central followed by East, North, North East and West being the least expensive

In light of this, it appears that there are other factors that plays a pivotal role in the price of the AirBnB Listing. The price is not solely dependent on the Region of the property listing but dependent on other factors like the property type and room type to determine the price of the listing.

Typically Central area might be able to fetch a higher price due to its proximity to the city and events / activities going on the area. Other factors such as seasonality, room size, amenities and more can also be taken into consideration which are not present in this data set as it can potentially impact the pricing algorithm in AirBnB

# data[].attributes.body
1. Predicting Airbnb prices with machine learning and location data https://towardsdatascience.com/predicting-airbnb-prices-with-machine-learning-and-location-data-5c1e033d0a5a

2. Hands-on with Feature Engineering Techniques: Transforming Variables
https://heartbeat.fritz.ai/hands-on-with-feature-engineering-techniques-transforming-variables-acea03472e24

3. Logs Transformation in a Regression Equation
http://www.stat.wharton.upenn.edu/~stine/stat621/handouts/LogsInRegression.pdf

# data[].attributes.headline
show me the MONEY ... log-transforming AIRBNB prices

# data[].attributes.subject
show me the MONEY ... log-transforming AIRBNB prices

# data[].attributes.body
Predictive modelling involves learning the relationships between a target outcome (dependent variable) and the predictors (independent variables) affecting it. In this case:

Target: the price an Airbnb host can charge for her apartment

Predictors: the apartment's location, furnishings, other listing attributes, etc.

To improve model accuracies and arrive at the best conclusions, Data Scientists spend much of their time pre-processing data to engineer the best features for use in training their machines.

We attempt to illustrate such a process here, with the price (target) variable; exploring the column in detail and applying log-transformation to handle the outliers found in the column. Tasks performed:

Profiling the price column to understand its measure of central tendency (mean, median) and its distribution (skewness, kurtosis)

Cleaning empty or missing values, replacing them with the median price

Transforming (log10) the cleaned price column in Tibco Spotfire, to re-shape its distribution to mirror a normal distribution approximately.

Re-profiling (iterative) the transformed price column to assess its fit for use in modelling.

Applying log-transformation to right-skewed data reduces the distances between data points (from equal to exponential distance) and re-shapes the data distribution to one that is more normally distributed. The Data Science community found the derived regression equations (i.e., in linear regression models) are better-fits of the data throughout the dataset (have smaller RMSEs) and lead to improved predictive powers of the models.

click picture for Tibco Spotfire documentations

# data[].attributes.headline
scrutinise the DATA and it will confess to anything

# data[].attributes.subject
scrutinise the DATA and it will confess to anything

# data[].attributes.body
Data on Airbnb listings in Singapore was obtained from insideairbnb.com, a free to use repository of Airbnb listings in various cities around the world.

Click worksheet names for data dictionaries
Airbnb listings (26 columns, 7908 aggregated rows of tall & skinny property listing data) -- the apartments, their furnishing, prices, and past review ratings, etc.

3700 listings (rows) from hosts who registered from 2009 to 2015

4208 listings from hosts who registered from 2016 to 2019

Airbnb neighbourhood (2 columns, 55 rows of categorical labels) -- the towns and regions where the apartments were located.

# data[].attributes.body
Temasek Polytechnic  School of Informatics & IT
CDS1C01 Augmented data preparation
AY2020/2021 October semester

Lim Qi Wen       2081769H@student.tp.edu.eg
Lim How Yong   2081766D@student.tp.edu.sg