nullTechniques of Data AnalysisTechniques of Data AnalysisAssoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman
Director
Centre for Real Estate Studies
Faculty of Engineering and Geoinformation Science
Universiti Tekbnologi Malaysia
Skudai, JohorObjectivesObjectives
Overall: Reinforce your understanding from the main lecture
Specific:
* Concepts of data analysis
* Some data analysis techniques
* Some tips for data analysis
What I will not do:
* To teach every bit and pieces of statistical analysis
techniquesData analysis – “The Concept”Data analysis – “The Concept”Approach to de-synthesizing data, informational, and/or factual elements to answer research questions
Method of putting together facts and figures
to solve research problem
Systematic process of utilizing data to address research questions
Breaking down research issues through utilizing controlled data and factual informationCategories of data analysisCategories of data analysisNarrative (e.g. laws, arts)
Descriptive (e.g. social sciences)
Statistical/mathematical (pure/applied sciences)
Audio-Optical (e.g. telecommunication)
Others
Most research analyses, arguably, adopt the first
three.
The second and third are, arguably, most popular
in pure, applied, and social sciences
Statistical MethodsStatistical MethodsSomething to do with “statistics”
Statistics: “meaningful” quantities about a sample of objects, things, persons, events, phenomena, etc.
Widely used in social sciences.
Simple to complex issues. E.g.
* correlation
* anova
* manova
* regression
* econometric modelling
Two main categories:
* Descriptive statistics
* Inferential statisticsDescriptive statisticsDescriptive statisticsUse sample information to explain/make abstraction of population “phenomena”.
Common “phenomena”:
* Association (e.g. σ1,2.3 = 0.75)
* Tendency (left-skew, right-skew)
* Causal relationship (e.g. if X, then, Y)
* Trend, pattern, dispersion, range
Used in non-parametric analysis (e.g. chi-square, t-test, 2-way anova)
Examples of “abstraction” of phenomenaExamples of “abstraction” of phenomenaExamples of “abstraction” of phenomenaExamples of “abstraction” of phenomena% prediction errorInferential statisticsInferential statisticsUsing sample statistics to infer some “phenomena” of population parameters
Common “phenomena”: cause-and-effect * One-way r/ship
* Multi-directional r/ship
* Recursive
Use parametric analysis
Y1 = f(Y2, X, e1)
Y2 = f(Y1, Z, e2)
Y1 = f(X, e1)
Y2 = f(Y1, Z, e2)
Y = f(X)Examples of relationshipExamples of relationshipDep=9t – 215.8Dep=7t – 192.6Which one to use?Which one to use?Nature of research
* Descriptive in nature?
* Attempts to “infer”, “predict”, find “cause-and-effect”,
“influence”, “relationship”?
* Is it both?
Research design (incl. variables involved). E.g.
Outputs/results expected
* research issue
* research questions
* research hypotheses
At post-graduate level research, failure to choose the correct data analysis technique is an almost sure ingredient for thesis failure.Common mistakes in data analysisCommon mistakes in data analysisWrong techniques. E.g.
Infeasible techniques. E.g.
How to design ex-ante effects of KLIA? Development occurs “before” and “after”! What is the control treatment?
Further explanation!
Abuse of statistics. E.g.
Simply exclude a techniqueNote: No way can Likert scaling show “cause-and-effect” phenomena!Common mistakes (contd.) – “Abuse of statistics”Common mistakes (contd.) – “Abuse of statistics”How to avoid mistakes - Useful tipsHow to avoid mistakes - Useful tipsCrystalize the research problem → operability of it!
Read literature on data analysis techniques.
Evaluate various techniques that can do similar things w.r.t. to research problem
Know what a technique does and what it doesn’t
Consult people, esp. supervisor
Pilot-run the data and evaluate results
Don’t do research??
Principles of analysisPrinciples of analysisGoal of an analysis:
* To explain cause-and-effect phenomena
* To relate research with real-world event
* To predict/forecast the real-world
phenomena based on research
* Finding answers to a particular problem
* Making conclusions about real-world event
based on the problem
* Learning a lesson from the problemPrinciples of analysis (contd.)Principles of analysis (contd.)
Data can’t “talk”
An analysis contains some aspects of scientific
reasoning/argument:
* Define
* Interpret
* Evaluate
* Illustrate
* Discuss
* Explain
* Clarify
* Compare
* ContrastPrinciples of analysis (contd.)Principles of analysis (contd.)An analysis must have four elements:
* Data/information (what)
* Scientific reasoning/argument (what?
who? where? how? what happens?)
* Finding (what results?)
* Lesson/conclusion (so what? so how?
therefore,…)
Example
Principles of data analysisPrinciples of data analysisBasic guide to data analysis:
* “Analyse” NOT “narrate”
* Go back to research flowchart
* Break down into research objectives and
research questions
* Identify phenomena to be investigated
* Visualise the “expected” answers
* Validate the answers with data
* Don’t tell something not supported by
dataPrinciples of data analysis (contd.)Principles of data analysis (contd.)More female shoppers than male shoppers
More young female shoppers than young male shoppers
Young male shoppers are not interested to shop at the shopping complexData analysis (contd.)Data analysis (contd.)When analysing:
* Be objective
* Accurate
* True
Separate facts and opinion
Avoid “wrong” reasoning/argument. E.g. mistakes in interpretation.
null
Introductory Statistics for Social Sciences
Basic concepts
Central tendency
Variability
Probability
Statistical Modelling
Basic ConceptsBasic ConceptsPopulation: the whole set of a “universe”
Sample: a sub-set of a population
Parameter: an unknown “fixed” value of population characteristic
Statistic: a known/calculable value of sample characteristic representing that of the population. E.g.
μ = mean of population, = mean of sample
Q: What is the mean price of houses in J.B.?
A: RM 210,000
J.B. houses
μ = ?SSTDSTSD1= 300,000= 120,0002= 210,0003Basic Concepts (contd.)Basic Concepts (contd.)Randomness: Many things occur by pure chances…rainfall, disease, birth, death,..
Variability: Stochastic processes bring in them various different dimensions, characteristics, properties, features, etc., in the population
Statistical analysis methods have been developed to deal with these very nature of real world.“Central Tendency”“Central Tendency”Central Tendency – “Mean”,Central Tendency – “Mean”,For individual observations, . E.g.
X = {3,5,7,7,8,8,8,9,9,10,10,12}
= 96 ; n = 12
Thus, = 96/12 = 8
The above observations can be organised into a frequency table and mean calculated on the basis of frequencies
= 96; = 12
Thus, = 96/12 = 8Central Tendency–“Mean of Grouped Data”Central Tendency–“Mean of Grouped Data”House rental or prices in the PMR are frequently tabulated as a range of values. E.g.
What is the mean rental across the areas?
= 23; = 3317.5
Thus, = 3317.5/23 = 144.24Central Tendency – “Median”Central Tendency – “Median”Let say house rentals in a particular town are tabulated as follows:
Calculation of “median” rental needs a graphical aids→
Median = (n+1)/2 = (25+1)/2 =13th. Taman
2. (i.e. between 10 – 15 points on the vertical axis of ogive).
3. Corresponds to RM 140-145/month on the horizontal axis
4. There are (17-8) = 9 Taman in the range of RM 140-145/month5. Taman 13th. is 5th. out of the 9
Taman
6. The interval width is 5
7. Therefore, the median rental can
be calculated as:
140 + (5/9 x 5) = RM 142.8Central Tendency – “Median” (contd.)Central Tendency – “Median” (contd.)Central Tendency – “Quartiles” (contd.)Central Tendency – “Quartiles” (contd.)Upper quartile = ¾(n+1) = 19.5th. Taman
UQ = 145 + (3/7 x 5) = RM 147.1/month
Lower quartile = (n+1)/4 = 26/4 = 6.5 th. Taman
LQ = 135 + (3.5/5 x 5) = RM138.5/month
Inter-quartile = UQ – LQ = 147.1 – 138.5 = 8.6th. Taman
IQ = 138.5 + (4/5 x 5) = RM 142.5/month
“Variability”“Variability”Indicates dispersion, spread, variation, deviation
For single population or sample data:
where σ2 and s2 = population and sample variance respectively, xi = individual observations, μ = population mean, = sample mean, and n = total number of individual observations.
The square roots are:
standard deviation standard deviation
“Variability” (contd.)“Variability” (contd.)Why “measure of dispersion” important?
Consider returns from two categories of shares:
* Shares A (%) = {1.8, 1.9, 2.0, 2.1, 3.6}
* Shares B (%) = {1.0, 1.5, 2.0, 3.0, 3.9}
Mean A = mean B = 2.28%
But, different variability!
Var(A) = 0.557, Var(B) = 1.367
* Would you invest in category A shares or
category B shares?“Variability” (contd.)“Variability” (contd.)Coefficient of variation – COV – std. deviation as % of the mean:
Could be a better measure compared to std. dev.
COV(A) = 32.73%, COV(B) = 51.28%
“Variability” (contd.)“Variability” (contd.)Std. dev. of a frequency distribution
The following table shows the age distribution of second-time home buyers:
x^“Probability Distribution”“Probability Distribution”Defined as of probability density function (pdf).
Many types: Z, t, F, gamma, etc.
“God-given” nature of the real world event.
General form:
E.g.(continuous)(discrete)“Probability Distribution” (contd.)“Probability Distribution” (contd.)“Probability Distribution” (contd.)“Probability Distribution” (contd.)Values of x are discrete (discontinuous)
Sum of lengths of vertical bars p(X=x) = 1 all xDiscrete valuesDiscrete values“Probability Distribution” (contd.)“Probability Distribution” (contd.)▪ Many real world phenomena
take a form of continuous
random variable
▪ Can take any values between
two limits (e.g. income, age,
weight, price, rental, etc.)“Probability Distribution” (contd.)“Probability Distribution” (contd.)P(Rental = RM 8) = 0 P(Rental < RM 3.00) = 0.206
P(Rental < RM7) = 0.972 P(Rental RM 4.00) = 0.544
P(Rental 7) = 0.028 P(Rental < RM 2.00) = 0.053“Probability Distribution” (contd.)“Probability Distribution” (contd.)Ideal distribution of such phenomena:
* Bell-shaped, symmetrical
* Has a function of
μ = mean of variable x
σ = std. dev. Of x
π = ratio of circumference of a
circle to its diameter = 3.14
e = base of natural log = 2.71828“Probability distribution” “Probability distribution” μ ± 1σ = ? = ____% from total observation
μ ± 2σ = ? = ____% from total observation
μ ± 3σ = ? = ____% from total observation“Probability distribution”“Probability distribution”* Has the following distribution of observation“Probability distribution”“Probability distribution”There are various other types and/or shapes of distribution. E.g.
Not “ideally” shaped like the previous one Note: p(AGE=age) ≠ 1
How to turn this graph into a probability distribution function (p.d.f.)?“Z-Distribution”“Z-Distribution”(X=x) is given by area under curve
Has no standard algebraic method of integration → Z ~ N(0,1)
It is called “normal distribution” (ND)
Standard reference/approximation of other distributions. Since there are various f(x) forming NDs, SND is needed
To transform f(x) into f(z):
x - µ
Z = --------- ~ N(0, 1)
σ
160 –155
E.g. Z = ------------- = 0.926
5.4
Probability is such a way that:
* Approx. 68% -1< z <1
* Approx. 95% -1.96 < z < 1.96
* Approx. 99% -2.58 < z < 2.58“Z-distribution” (contd.)“Z-distribution” (contd.)When X= μ, Z = 0, i.e.
When X = μ + σ, Z = 1
When X = μ + 2σ, Z = 2
When X = μ + 3σ, Z = 3 and so on.
It can be proven that P(X1 0.811)=0.1867
P(145,000 “30-34”)
(AGE ≤ 20-24)
( “35-39”≤ AGE < “50-54”)Test yourselves!Test yourselves!Q6: You are asked by a property marketing manager to ascertain whether
or not distance to work and distance to the city are “equally” important
factors influencing people’s choice of house location.
You are given the following data for the purpose of testing:
Explore the data as follows:
Create histograms for both distances. Comment on the shape of the histograms. What is you conclusion?
Construct scatter diagram of both distances. Comment on the output.
Explore the data and give some analysis.
Set a hypothesis that means of both distances are the same. Make your conclusion.
Test yourselves! (contd.)Test yourselves! (contd.)Q7: From your initial investigation, you belief that tenants of
“low-quality” housing choose to rent particular flat units just
to find shelters. In this context ,these groups of people do
not pay much attention to pertinent aspects of “quality
life” such as accessibility, good surrounding, security, and
physical facilities in the living areas.
(a) Set your research design and data analysis procedure to address
the research issue
(b) Test your hypothesis that low-income tenants do not perceive “quality life” to be important in paying their house rentals.
Thank you Thank you
本文档为【060_Techniques_of_Data_Analysis】,请使用软件OFFICE或WPS软件打开。作品中的文字与图均可以修改和编辑,
图片更改请在作品中右键图片并更换,文字修改请直接点击文字进行修改,也可以新增和删除文档中的内容。
该文档来自用户分享,如有侵权行为请发邮件ishare@vip.sina.com联系网站客服,我们会及时删除。
[版权声明] 本站所有资料为用户分享产生,若发现您的权利被侵害,请联系客服邮件isharekefu@iask.cn,我们尽快处理。
本作品所展示的图片、画像、字体、音乐的版权可能需版权方额外授权,请谨慎使用。
网站提供的党政主题相关内容(国旗、国徽、党徽..)目的在于配合国家政策宣传,仅限个人学习分享使用,禁止用于任何广告和商用目的。