Commit ee3c40ac authored by François Févotte's avatar François Févotte

Ex 4-1: meilleure manière de traiter les poids dans la régression

parent b1342638
...@@ -773,9 +773,9 @@ Status `~/tmp/MOOC-RR/module4/Project.toml` ...@@ -773,9 +773,9 @@ Status `~/tmp/MOOC-RR/module4/Project.toml`
<div id="8196e83b-fcfd-404d-a407-99bab186c19a" style="width:576px;height:384px;"></div> <div id="7db6b95a-3f30-4f91-8d9e-d26a8f7a5f59" style="width:576px;height:384px;"></div>
<script> <script>
PLOT = document.getElementById('8196e83b-fcfd-404d-a407-99bab186c19a'); PLOT = document.getElementById('7db6b95a-3f30-4f91-8d9e-d26a8f7a5f59');
Plotly.plot(PLOT, [ Plotly.plot(PLOT, [
{ {
"xaxis": "x1", "xaxis": "x1",
...@@ -1003,7 +1003,7 @@ Status &#96;~/tmp/MOOC-RR/module4/Project.toml&#96; ...@@ -1003,7 +1003,7 @@ Status &#96;~/tmp/MOOC-RR/module4/Project.toml&#96;
<pre class="output"> <pre class="output">
StatsModels.TableRegressionModel&#123;GeneralizedLinearModel&#123;GLM.GlmResp&#123;Array&#123;Float64,1&#125;,Binomial&#123;Float64&#125;,LogitLink&#125;,GLM.DensePredChol&#123;Float64,LinearAlgebra.Cholesky&#123;Float64,Array&#123;Float64,2&#125;&#125;&#125;&#125;,Array&#123;Float64,2&#125;&#125; StatsModels.TableRegressionModel&#123;GLM.GeneralizedLinearModel&#123;GLM.GlmResp&#123;Array&#123;Float64,1&#125;,Distributions.Binomial&#123;Float64&#125;,GLM.LogitLink&#125;,GLM.DensePredChol&#123;Float64,LinearAlgebra.Cholesky&#123;Float64,Array&#123;Float64,2&#125;&#125;&#125;&#125;,Array&#123;Float64,2&#125;&#125;
Frequency ~ 1 &#43; Temperature Frequency ~ 1 &#43; Temperature
...@@ -1018,22 +1018,15 @@ Temperature -0.115601 0.115184 -1.00362 0.3156 -0.341358 0.110156 ...@@ -1018,22 +1018,15 @@ Temperature -0.115601 0.115184 -1.00362 0.3156 -0.341358 0.110156
<p>The maximum likelyhood estimator of the intercept and of Temperature are thus <span class="math">$\hat{\alpha}=5.085$</span> and <span class="math">$\hat{\beta}=-0.116$</span>.</p> <p>The maximum likelyhood estimator of the intercept and of Temperature are thus <span class="math">$\hat{\alpha}=5.085$</span> and <span class="math">$\hat{\beta}=-0.116$</span>.</p>
<p>This corresponds to the values from the article of Dalal et al. The standard errors are <span class="math">$s_{\hat{\alpha}} = 7.477$</span> and <span class="math">$s_{\hat{\beta}} = 0.115$</span>, which is different from the <span class="math">$3.052$</span> and <span class="math">$0.047$</span> reported by Dallal et al. The deviance is <span class="math">$G^2 = 3.014$</span> with 22 degrees of freedom.</p> <p>This corresponds to the values from the article of Dalal et al. The standard errors are <span class="math">$s_{\hat{\alpha}} = 7.477$</span> and <span class="math">$s_{\hat{\beta}} = 0.115$</span>, which is different from the <span class="math">$3.052$</span> and <span class="math">$0.047$</span> reported by Dallal et al.</p>
<p>I cannot find any value similar to the Goodness of fit &#40;<span class="math">$G^2=18.086$</span>&#41; reported by Dalal et al. However, the number of degrees of freedom is similar to theirs &#40;21&#41;.</p> <p>The deviance is <span class="math">$G^2 = 3.014$</span> with 22 degrees of freedom. I cannot find any value similar to the Goodness of fitreported by Dalal <em>et al.</em> &#40;<span class="math">$G^2=18.086$</span>&#41;. However, the number of degrees of freedom is different but at least similar to theirs &#40;21&#41;.</p>
<p>There seems to be something wrong. Oh I know, I haven&#39;t indicated that my observations are actually the result of 6 observations for each rocket launch. The correct way to do this would be to weight the data using the <code>Count</code> column. Since I don&#39;t know how to do that with the <a href="https://github.com/JuliaStats/GLM.jl">GLM</a> package I&#39;m using, I will simply duplicate the data:</p> <p>There seems to be something wrong. Oh I know, I haven&#39;t indicated that my observations are actually the result of 6 observations for each rocket launch. Let&#39;s indicate these weights &#40;since the weights are always the same throughout all experiments, it does not change the estimates of the fit but it does influence de variance estimate&#41;.</p>
<pre class='hljl'> <pre class='hljl'>
<span class='hljl-n'>weighted_data</span><span class='hljl-t'> </span><span class='hljl-oB'>=</span><span class='hljl-t'> </span><span class='hljl-nf'>DataFrame</span><span class='hljl-p'>(</span><span class='hljl-n'>Temperature</span><span class='hljl-oB'>=</span><span class='hljl-n'>Int</span><span class='hljl-p'>[],</span><span class='hljl-t'> </span><span class='hljl-n'>Frequency</span><span class='hljl-oB'>=</span><span class='hljl-n'>Float64</span><span class='hljl-p'>[])</span><span class='hljl-t'> <span class='hljl-n'>model</span><span class='hljl-t'> </span><span class='hljl-oB'>=</span><span class='hljl-t'> </span><span class='hljl-nf'>glm</span><span class='hljl-p'>(</span><span class='hljl-nd'>@formula</span><span class='hljl-p'>(</span><span class='hljl-n'>Frequency</span><span class='hljl-t'> </span><span class='hljl-oB'>~</span><span class='hljl-t'> </span><span class='hljl-n'>Temperature</span><span class='hljl-p'>),</span><span class='hljl-t'> </span><span class='hljl-n'>data</span><span class='hljl-p'>,</span><span class='hljl-t'>
</span><span class='hljl-k'>for</span><span class='hljl-t'> </span><span class='hljl-n'>row</span><span class='hljl-t'> </span><span class='hljl-kp'>in</span><span class='hljl-t'> </span><span class='hljl-nf'>eachrow</span><span class='hljl-p'>(</span><span class='hljl-n'>data</span><span class='hljl-p'>)</span><span class='hljl-t'> </span><span class='hljl-nf'>Binomial</span><span class='hljl-p'>(),</span><span class='hljl-t'> </span><span class='hljl-nf'>LogitLink</span><span class='hljl-p'>();</span><span class='hljl-t'>
</span><span class='hljl-k'>for</span><span class='hljl-t'> </span><span class='hljl-n'>_</span><span class='hljl-t'> </span><span class='hljl-kp'>in</span><span class='hljl-t'> </span><span class='hljl-ni'>1</span><span class='hljl-oB'>:</span><span class='hljl-n'>row</span><span class='hljl-oB'>.</span><span class='hljl-n'>Count</span><span class='hljl-t'> </span><span class='hljl-n'>wts</span><span class='hljl-oB'>=</span><span class='hljl-n'>data</span><span class='hljl-oB'>.</span><span class='hljl-n'>Count</span><span class='hljl-p'>)</span><span class='hljl-t'>
</span><span class='hljl-nf'>push!</span><span class='hljl-p'>(</span><span class='hljl-n'>weighted_data</span><span class='hljl-p'>,</span><span class='hljl-t'> </span><span class='hljl-p'>(</span><span class='hljl-n'>Temperature</span><span class='hljl-oB'>=</span><span class='hljl-n'>row</span><span class='hljl-oB'>.</span><span class='hljl-n'>Temperature</span><span class='hljl-p'>,</span><span class='hljl-t'>
</span><span class='hljl-n'>Frequency</span><span class='hljl-oB'>=</span><span class='hljl-n'>row</span><span class='hljl-oB'>.</span><span class='hljl-n'>Frequency</span><span class='hljl-p'>))</span><span class='hljl-t'>
</span><span class='hljl-k'>end</span><span class='hljl-t'>
</span><span class='hljl-k'>end</span><span class='hljl-t'>
</span><span class='hljl-n'>model</span><span class='hljl-t'> </span><span class='hljl-oB'>=</span><span class='hljl-t'> </span><span class='hljl-nf'>glm</span><span class='hljl-p'>(</span><span class='hljl-nd'>@formula</span><span class='hljl-p'>(</span><span class='hljl-n'>Frequency</span><span class='hljl-t'> </span><span class='hljl-oB'>~</span><span class='hljl-t'> </span><span class='hljl-n'>Temperature</span><span class='hljl-p'>),</span><span class='hljl-t'> </span><span class='hljl-n'>weighted_data</span><span class='hljl-p'>,</span><span class='hljl-t'>
</span><span class='hljl-nf'>Binomial</span><span class='hljl-p'>(),</span><span class='hljl-t'> </span><span class='hljl-nf'>LogitLink</span><span class='hljl-p'>())</span><span class='hljl-t'>
</span><span class='hljl-n'>α</span><span class='hljl-p'>,</span><span class='hljl-t'> </span><span class='hljl-n'>β</span><span class='hljl-t'> </span><span class='hljl-oB'>=</span><span class='hljl-t'> </span><span class='hljl-nf'>coef</span><span class='hljl-p'>(</span><span class='hljl-n'>model</span><span class='hljl-p'>)</span><span class='hljl-t'> </span><span class='hljl-n'>α</span><span class='hljl-p'>,</span><span class='hljl-t'> </span><span class='hljl-n'>β</span><span class='hljl-t'> </span><span class='hljl-oB'>=</span><span class='hljl-t'> </span><span class='hljl-nf'>coef</span><span class='hljl-p'>(</span><span class='hljl-n'>model</span><span class='hljl-p'>)</span><span class='hljl-t'>
</span><span class='hljl-n'>σα</span><span class='hljl-p'>,</span><span class='hljl-t'> </span><span class='hljl-n'>σβ</span><span class='hljl-t'> </span><span class='hljl-oB'>=</span><span class='hljl-t'> </span><span class='hljl-nf'>stderror</span><span class='hljl-p'>(</span><span class='hljl-n'>model</span><span class='hljl-p'>)</span><span class='hljl-t'> </span><span class='hljl-n'>σα</span><span class='hljl-p'>,</span><span class='hljl-t'> </span><span class='hljl-n'>σβ</span><span class='hljl-t'> </span><span class='hljl-oB'>=</span><span class='hljl-t'> </span><span class='hljl-nf'>stderror</span><span class='hljl-p'>(</span><span class='hljl-n'>model</span><span class='hljl-p'>)</span><span class='hljl-t'>
...@@ -1046,7 +1039,7 @@ Temperature -0.115601 0.115184 -1.00362 0.3156 -0.341358 0.110156 ...@@ -1046,7 +1039,7 @@ Temperature -0.115601 0.115184 -1.00362 0.3156 -0.341358 0.110156
<pre class="output"> <pre class="output">
StatsModels.TableRegressionModel&#123;GeneralizedLinearModel&#123;GLM.GlmResp&#123;Array&#123;Float64,1&#125;,Binomial&#123;Float64&#125;,LogitLink&#125;,GLM.DensePredChol&#123;Float64,LinearAlgebra.Cholesky&#123;Float64,Array&#123;Float64,2&#125;&#125;&#125;&#125;,Array&#123;Float64,2&#125;&#125; StatsModels.TableRegressionModel&#123;GLM.GeneralizedLinearModel&#123;GLM.GlmResp&#123;Array&#123;Float64,1&#125;,Distributions.Binomial&#123;Float64&#125;,GLM.LogitLink&#125;,GLM.DensePredChol&#123;Float64,LinearAlgebra.Cholesky&#123;Float64,Array&#123;Float64,2&#125;&#125;&#125;&#125;,Array&#123;Float64,2&#125;&#125;
Frequency ~ 1 &#43; Temperature Frequency ~ 1 &#43; Temperature
...@@ -1054,14 +1047,14 @@ Coefficients: ...@@ -1054,14 +1047,14 @@ Coefficients:
───────────────────────────────────────────────────────────────────────────── ─────────────────────────────────────────────────────────────────────────────
Estimate Std. Error z value Pr&#40;&gt;|z|&#41; Lower 95&#37; Upper 95&#37; Estimate Std. Error z value Pr&#40;&gt;|z|&#41; Lower 95&#37; Upper 95&#37;
───────────────────────────────────────────────────────────────────────────── ─────────────────────────────────────────────────────────────────────────────
&#40;Intercept&#41; 5.08498 3.05248 1.66585 0.0957 -0.897782 11.0677 &#40;Intercept&#41; 5.08498 3.05247 1.66585 0.0957 -0.897762 11.0677
Temperature -0.115601 0.0470238 -2.45835 0.0140 -0.207766 -0.0234362 Temperature -0.115601 0.0470236 -2.45836 0.0140 -0.207766 -0.0234366
───────────────────────────────────────────────────────────────────────────── ─────────────────────────────────────────────────────────────────────────────
</pre> </pre>
<p>Good, now I have recovered the asymptotic standard errors <span class="math">$s_{\hat{\alpha}} = 3.052$</span> and <span class="math">$s_{\hat{\beta}} = 0.047$</span>,</p> <p>Good, now I have recovered the asymptotic standard errors <span class="math">$s_{\hat{\alpha}} = 3.052$</span> and <span class="math">$s_{\hat{\beta}} = 0.047$</span>,</p>
<p>The Goodness of fit &#40;Deviance&#41; indicated for this model is <span class="math">$G^2 = 18.086$</span> with 137 degrees of freedom. Now <span class="math">$G^2$</span> is in good accordance to the results of the Dalal <em>et al.</em> article, but the number of degrees of freedom is 6 times larger than i should, due to my tampering of the data to duplicate them instead of weighting them.</p> <p>The Goodness of fit &#40;Deviance&#41; indicated for this model is <span class="math">$G^2 = 18.086$</span> with 137 degrees of freedom. Now <span class="math">$G^2$</span> is in good accordance to the results of the Dalal <em>et al.</em> article, but the number of degrees of freedom is approximately 6 times larger than that of Dalal <em>et al</em>. Note that, even removing this factor &#40;which is probably due to the way the number of residual degrees of freedom are defined in both libraries in the presence of weights&#41;, the values are similar but still differ by 9&#37;.</p>
<h1>Predicting failure probability</h1> <h1>Predicting failure probability</h1>
<p>The temperature when launching the shuttle was 31°F. Let&#39;s try to estimate the failure probability for such temperature using our model:</p> <p>The temperature when launching the shuttle was 31°F. Let&#39;s try to estimate the failure probability for such temperature using our model:</p>
...@@ -1078,9 +1071,9 @@ Temperature -0.115601 0.0470238 -2.45835 0.0140 -0.207766 -0.0234362 ...@@ -1078,9 +1071,9 @@ Temperature -0.115601 0.0470238 -2.45835 0.0140 -0.207766 -0.0234362
<div id="e862177e-031e-4363-8515-b6f4128e30a1" style="width:576px;height:384px;"></div> <div id="3b71fccf-f2fd-4b5f-abd3-669b57954ac9" style="width:576px;height:384px;"></div>
<script> <script>
PLOT = document.getElementById('e862177e-031e-4363-8515-b6f4128e30a1'); PLOT = document.getElementById('3b71fccf-f2fd-4b5f-abd3-669b57954ac9');
Plotly.plot(PLOT, [ Plotly.plot(PLOT, [
{ {
"xaxis": "x1", "xaxis": "x1",
...@@ -1417,247 +1410,247 @@ Temperature -0.115601 0.0470238 -2.45835 0.0140 -0.207766 -0.0234362 ...@@ -1417,247 +1410,247 @@ Temperature -0.115601 0.0470238 -2.45835 0.0140 -0.207766 -0.0234362
"width": 1 "width": 1
}, },
"y": [ "y": [
0.8343729945321491, 0.8343729944887327,
0.8303404395482706, 0.8303404395044195,
0.8262301507742621, 0.8262301507299863,
0.8220416250063012, 0.8220416249616112,
0.817774406282189, 0.8177744062370959,
0.8134280882973987, 0.8134280882519144,
0.8090023168250383, 0.8090023167791754,
0.804496792132775, 0.8044967920865465,
0.7999112713893656, 0.7999112713427853,
0.7952455710530575, 0.7952455710061399,
0.7904995692337479, 0.7904995691865084,
0.7856732080204392, 0.7856732079728936,
0.7807664957651925, 0.7807664957173572,
0.7757795093144834, 0.7757795092663756,
0.7707123961785791, 0.7707123961302167,
0.7655653766293189, 0.76556537658072,
0.7603387457164711, 0.7603387456676551,
0.7550328751926784, 0.7550328751436648,
0.7496482153368779, 0.7496482152876869,
0.744185296666015, 0.7441852966166677,
0.7386447315248507, 0.7386447314753681,
0.733027215543692, 0.7330272154940964,
0.7273335289539781, 0.7273335289042917,
0.7215645377517999, 0.7215645377020458,
0.7157211946996577, 0.7157211946498594,
0.7098045401570411, 0.7098045401072225,
0.7038157027307677, 0.7038157026809528,
0.6977558997364355, 0.6977558996866491,
0.6916264374628388, 0.691626437413106,
0.6854287112317441, 0.6854287111820898,
0.6791642052460566, 0.6791642051965066,
0.6728344922200988, 0.672834492170679,
0.666441232786479, 0.6664412327372151,
0.6599861746748452, 0.6599861746257635,
0.6534711516587098, 0.6534711516098363,
0.6468980822674514, 0.6468980822188125,
0.6402689682615972, 0.6402689682132191,
0.6335858928705154, 0.6335858928224241,
0.6268510187927153, 0.6268510187449374,
0.6200665859600667, 0.6200665859126281,
0.6132349090683641, 0.6132349090212904,
0.6063583748778226, 0.6063583748311396,
0.5994394392882448, 0.5994394392419781,
0.5924806241947568, 0.5924806241489308,
0.5854845141311654, 0.585484514085805,
0.5784537527091304, 0.5784537526642599,
0.5713910388624514, 0.5713910388180944,
0.5642991229068514, 0.5642991228630314,
0.5571808024266807, 0.5571808023834203,
0.5500389180009383, 0.5500389179582594,
0.5428763487819457, 0.5428763487398697,
0.5356960079408601, 0.5356960078994081,
0.5285008379949963, 0.5285008379541878,
0.5212938060326261, 0.5212938059924808,
0.5140778988515446, 0.514077898812081,
0.5068561180281956, 0.5068561179894309,
0.4996314749345776, 0.4996314748965288,
0.4924069857204715, 0.492406985683154,
0.4851856662787247, 0.48518566624215403,
0.4779705272114517, 0.47797052717564165,
0.47076456881498563, 0.47076456877994904,
0.46357077610131336, 0.46357077606706254,
0.4563921138735051, 0.45639211384005113,
0.44923152187231224, 0.44923152183966486,
0.44209191001069065, 0.44209190997885905,
0.4349761537124744, 0.4349761536814667,
0.4278870893707977, 0.42788708934062053,
0.4208275099411642, 0.42082750991182377,
0.4138001606832668, 0.413800160654768,
0.40680773506479373, 0.40680773503714013,
0.39985287083952786, 0.3998528708127225,
0.3929381463110585, 0.3929381462851031,
0.3860660767923737, 0.3860660767672689,
0.37923911127052695, 0.37923911124627285,
0.37245962928445564, 0.37245962926105114,
0.36572993802288195, 0.36572993800032483,
0.3590522696480814, 0.3590522696263686,
0.35242877885013985, 0.35242877882926754,
0.34586154063516156, 0.3458615406151246,
0.3393525483497468, 0.3393525483305398,
0.3329037119429306, 0.33290371192454693,
0.3265168564656702, 0.3265168564481027,
0.3201937208069072, 0.32019372079014785,
0.31393595666419893, 0.3139359566482387,
0.3077451277459338, 0.30774512773076346,
0.30162270920122125, 0.3016227091868303,
0.2955700872726619, 0.2955700872590396,
0.2895885591664015, 0.28958855915353676,
0.2836793331331148, 0.2836793331209955,
0.2778435287528768, 0.27784352874149043,
0.2720821774162679, 0.27208217740560153,
0.26639622299350313, 0.26639622298354354,
0.2607865226828959, 0.26078652267362884,
0.25525384802955275, 0.2552538480209643,
0.249798886104856, 0.24979888609693163,
0.24442224083700642, 0.24442224082973107,
0.2391244344826909, 0.23912443447604972,
0.23390590922979299, 0.23390590922377055,
0.22876702892096915, 0.22876702891554954,
0.22370808088789118, 0.22370808088305907,
0.21872927788597865, 0.2187292778817181,
0.21383076011951677, 0.2138307601158116,
0.2090125973471875, 0.20901259734402192,
0.20427479105820717, 0.20427479105556492,
0.19961727670947108, 0.19961727670733612,
0.1950399260143586, 0.1950399260127147,
0.19054254927412537, 0.1905425492729564,
0.18612489774312097, 0.18612489774241112,
0.18178666601940532, 0.18178666601913862,
0.1775274944526878, 0.17752749445284824,
0.17334697156189055, 0.17334697156246254,
0.16924463645502494, 0.1692446364559929,
0.16521998124446513, 0.16521998124581339,
0.16127245345111377, 0.16127245345282729,
0.15740145839136546, 0.15740145839342923,
0.15360636154118626, 0.15360636154358534,
0.1498864908720423, 0.14988649087476216,
0.1462411391538237, 0.14624113915684997,
0.14266956622031157, 0.14266956622363003,
0.13917100119313955, 0.13917100119673628,
0.13574464466059075, 0.13574464466445227,
0.13238967080795028, 0.13238967081206315,
0.12910522949650427, 0.12910522950085554,
0.12589044828863363, 0.12589044829321056,
0.12274443441679084, 0.12274443442158084,
0.11966627669447896, 0.11966627669946997,
0.11665504736766594, 0.1166550473728459,
0.11370980390536166, 0.1137098039107192,
0.11082959072837197, 0.11082959073389588,
0.10801344087550532, 0.10801344088118454,
0.10526037760675964, 0.10526037761258362,
0.10256941594325082, 0.10256941594920928,
0.09993956414385886, 0.09993956414994169,
0.09736982511877146, 0.09736982512496896,
0.09485919778029053, 0.09485919778659342,
0.09240667833143555, 0.09240667833783464,
0.09001126149303501, 0.09001126149952163,
0.08767194167013932, 0.08767194167670497,
0.08538771405871287, 0.08538771406534931,
0.0831575756936799, 0.08315757570037932,
0.08098052643949967, 0.08098052644625442,
0.07885556992453378, 0.07885556993133656,
0.07678171442054815, 0.07678171442739198,
0.07475797366875855, 0.0747579736756367,
0.0727833676538836, 0.07278336766078954,
0.07085692332771698, 0.07085692333464455,
0.06897767528376893, 0.06897767529071216,
0.06714466638455384, 0.06714466639150697,
0.06535694834312447, 0.06535694835008216,
0.0636135822604664, 0.06361358226742339,
0.06191363912037254, 0.06191363912732391,
0.06025620024342098, 0.060256200250362026,
0.05864035770167245, 0.05864035770859863,
0.05706521469569536, 0.057065214702602424,
0.05552988589551237, 0.0555298859023962,
0.05403349774704249, 0.054033497753899255,
0.052575188745591785, 0.05257518875241788,
0.05115410967792009, 0.051154109684712006,
0.04976942383438114, 0.0497694238411357,
0.04842030719260539, 0.048420307199319425,
0.0471059485741575, 0.04710594858082812,
0.04582554977556969, 0.04582554978219424,
0.04457832567511219, 0.044578325681688034,
0.043363504316624515, 0.04336350432314926,
0.0421803269716941, 0.0421803269781655,
0.04102804818142671, 0.04102804818784267,
0.039905935779012996, 0.03990593578537159,
0.038813270894256194, 0.03881327090055567,
0.03774934794118315, 0.037749347947421776,
0.036713474589819994, 0.03671347459599624,
0.03570497172317438, 0.03570497172928688,
0.03472317338042357, 0.03472317338647097,
0.03376742668726787, 0.033767426693249,
0.032837091774370254, 0.0328370917802841,
0.03193154168476222, 0.03193154169060775,
0.031050162271057585, 0.03105016227683395,
0.030192352083279427, 0.030192352088985915,
0.02935752224806677, 0.029357522253702646,
0.028545096339991714, 0.02854509634555636,
0.0277545102456837, 0.027754510251176718,
0.026985212021421776, 0.026985212026842696,
0.026236661744822722, 0.02623666175017121,
0.025508331361221583, 0.025508331366497418,
0.024799704525308587, 0.024799704530511526,
0.024110276438556306, 0.02411027644368622,
0.023439553682942635, 0.02343955368799947,
0.022787054051445665, 0.02278705405642945,
0.022152306375760253, 0.022152306380671002,
0.02153485035165899, 0.021534850356496812,
0.02093423636239639, 0.020934236367161477,
0.020350025300529845, 0.020350025305222372,
0.019781788388507835, 0.019781788393128052,
0.019229106998354638, 0.019229107002902843,
0.018691572470758495, 0.018691572475235025,
0.018168785933850056, 0.018168785938255286,
0.01766035812193932, 0.017660358126273694,
0.017165909194459938, 0.01716590919872387,
0.01668506855535212, 0.01668506855954609,
0.0162174746730995, 0.016217474677224036,
0.015762774901618322, 0.015762774905673935,
0.015320625302182434, 0.015320625306169688,
0.014890690466553641, 0.014890690470473141,
0.014472643341472706, 0.014472643345325033,
0.01406616505465329, 0.014066165058439073,
0.013670944742409543, 0.013670944746129446,
0.013286679379035727, 0.013286679382690396,
0.01291307360804536, 0.012913073611635446,
0.012549839575367538, 0.012549839578893782,
0.012196696764587796, 0.012196696768050878,
0.011853371834311528, 0.011853371837712172,
0.011519598457720067, 0.01151959846105902,
0.011195117164380578, 0.011195117167658558,
0.010879675184363433, 0.010879675187581185,
0.010573026294713973, 0.010573026297872275,
0.010274930668318326, 0.010274930671417916,
0.009985154725196585, 0.00998515472823823,
0.009703470986251296, 0.009703470989235782,
0.009429657929493154, 0.009429657932421238,
0.009163499848760623, 0.009163499851633082,
0.00890478671494593, 0.008904786717763558,
0.008653314039734879, 0.008653314042498424,
0.008408882741863815, 0.008408882744574074,
0.008171299015893661, 0.008171299018551426,
0.007940374203496888, 0.007940374206102926,
0.007715924667250078, 0.007715924669805165,
0.007497771666922003, 0.00749777166942693,
0.007285741238244001, 0.007285741240699527,
0.007079664074146817, 0.007079664076553713,
0.0068793754084461355, 0.00687937541080518,
0.006684714901956439, 0.006684714904268383,
0.006495526531010921, 0.0064955265332765225,
0.006311658478363718, 0.006311658480583734,
0.0061329630264486765, 0.006132963028623865,
0.00595929645296774, 0.005959296455098823,
0.00579051892878043, 0.005790518930868158,
0.005626494418065105, 0.005626494420110219,
0.00546709058072131, 0.005467090582724528,
0.005312178676981671, 0.00531217867894371,
0.0051616334742012205, 0.005161633476122803,
0.005015333155791099, 0.005015333157672923,
0.004873159232262993 0.004873159234105756
], ],
"type": "scatter" "type": "scatter"
} }
...@@ -1742,8 +1735,8 @@ Temperature -0.115601 0.0470238 -2.45835 0.0140 -0.207766 -0.0234362 ...@@ -1742,8 +1735,8 @@ Temperature -0.115601 0.0470238 -2.45835 0.0140 -0.207766 -0.0234362
"visible": true, "visible": true,
"ticks": "inside", "ticks": "inside",
"range": [ "range": [
-0.02503118983596447, -0.02503118983466198,
0.8594041843681135 0.8594041843233947
], ],
"domain": [ "domain": [
0.07897368948673088, 0.07897368948673088,
......
...@@ -99,32 +99,23 @@ This corresponds to the values from the article of Dalal et al. The standard ...@@ -99,32 +99,23 @@ This corresponds to the values from the article of Dalal et al. The standard
errors are errors are
$s_{\hat{\alpha}} = `j @printf "%.3f" σα`$ and $s_{\hat{\alpha}} = `j @printf "%.3f" σα`$ and
$s_{\hat{\beta}} = `j @printf "%.3f" σβ`$, $s_{\hat{\beta}} = `j @printf "%.3f" σβ`$,
which is different from the $3.052$ and $0.047$ reported by Dallal et al. The which is different from the $3.052$ and $0.047$ reported by Dallal et al.
deviance is
$G^2 = `j @printf "%.3f" G²`$ with `j nDOF` degrees of freedom.
I cannot find any value similar to the Goodness of fit ($G^2=18.086$) reported The deviance is $G^2 = `j @printf "%.3f" G²`$ with `j nDOF` degrees of freedom.
by Dalal et al. However, the number of degrees of freedom is similar to theirs I cannot find any value similar to the Goodness of fitreported by Dalal *et al.*
(21). ($G^2=18.086$). However, the number of degrees of freedom is different but at
least similar to theirs (21).
There seems to be something wrong. Oh I know, I haven't indicated that my There seems to be something wrong. Oh I know, I haven't indicated that my
observations are actually the result of 6 observations for each rocket observations are actually the result of 6 observations for each rocket
launch. The correct way to do this would be to weight the data using the `Count` launch. Let's indicate these weights (since the weights are always the same
column. Since I don't know how to do that with the throughout all experiments, it does not change the estimates of the fit but it
[GLM](https://github.com/JuliaStats/GLM.jl) package I'm using, I will simply does influence de variance estimate).
duplicate the data:
```julia; wrap=false; hold=true ```julia; wrap=false; hold=true
weighted_data = DataFrame(Temperature=Int[], Frequency=Float64[]) model = glm(@formula(Frequency ~ Temperature), data,
for row in eachrow(data) Binomial(), LogitLink();
for _ in 1:row.Count wts=data.Count)
push!(weighted_data, (Temperature=row.Temperature,
Frequency=row.Frequency))
end
end
model = glm(@formula(Frequency ~ Temperature), weighted_data,
Binomial(), LogitLink())
α, β = coef(model) α, β = coef(model)
σα, σβ = stderror(model) σα, σβ = stderror(model)
...@@ -142,8 +133,11 @@ $s_{\hat{\beta}} = `j @printf "%.3f" σβ`$, ...@@ -142,8 +133,11 @@ $s_{\hat{\beta}} = `j @printf "%.3f" σβ`$,
The Goodness of fit (Deviance) indicated for this model is The Goodness of fit (Deviance) indicated for this model is
$G^2 = `j @printf "%.3f" G²`$ with `j nDOF` degrees of freedom. Now $G^2$ is in $G^2 = `j @printf "%.3f" G²`$ with `j nDOF` degrees of freedom. Now $G^2$ is in
good accordance to the results of the Dalal *et al.* article, but the number of good accordance to the results of the Dalal *et al.* article, but the number of
degrees of freedom is 6 times larger than i should, due to my tampering of the degrees of freedom is approximately 6 times larger than that of Dalal *et
data to duplicate them instead of weighting them. al*. Note that, even removing this factor (which is probably due to the way the
number of residual degrees of freedom are defined in both libraries in the
presence of weights), the values are similar but still differ by
`j @printf "%2.0f" 100 * (nDOF/6/21 - 1)`%.
# Predicting failure probability # Predicting failure probability
......
No preview for this file type
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment