-
Notifications
You must be signed in to change notification settings - Fork 0
/
research.html
8431 lines (8413 loc) · 578 KB
/
research.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html>
<head>
<!--Import materialize.css-->
<link type="text/css" rel="stylesheet" href="css/materialize.min.css" media="screen,projection"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"/>
<!--Dynamically Import external HTML.css-->
<!--Local Styling-->
<style>
img {
max-width: 100%;
max-height: 100%;
}
div.graph {
margin:0 auto;
width: 855px;
height: 200px;
}
div.main {
padding-left: 5%;
padding-right: 5%;
}
div.note {
margin: 10px;
padding-left: 50px;
padding-right: 50px;
}
div.collapsible-header {
font-weight: bold;
}
.numbering {
text-align: center;
margin-right: 1rem;
width: 2rem;
display: inline-block;
}
p.blurb {
text-align: justify;
text-justify: inter-word;
padding-left: 90px;
padding-right: 90px;
}
ul.blurb {
text-align: justify;
text-justify: inter-word;
padding-left: 90px;
padding-right: 90px;
}
span.cdates {
color: LightSlateGray;
font-weight: normal;
}
</style>
</head>
<body>
<!--Site Navigation-->
<nav>
<div class="nav-wrapper light-blue darken-2">
<ul id="nav-mobile" class="right hide-on-med-and-down">
<li><a href="index.html">Motivation</a></li>
<li><a href="index.html">Intermediate Results</a></li>
<li class="active"><a href="research.html">Research</a></li>
<li><a href="opensource.html">Source Code</a></li>
</ul>
</div>
</nav>
<p class="blurb"><b>The scatter plot on my <a href="http://unclassicallytrained.com">home page</a> is my attempt to answer the seemingly harmless question "Have popular songs become less unique?" But don't be fooled! That is a demon question - answering it was anything but easy. You see making that chart meant I first had to overcome challenges in data collection, correction, storage, organization, and communication. In fact answering that question required breaking it up into over 508 smaller questions.
One by one I wrestled those and, as if creating a personal Stackoverflow, I took note of each answer. This page is the record of that work. For tour guiding reasons I've included the big picture questions directly below; </p>
<ul class="blurb">
<li>- <a href="#517">What is this project's scope?</a></li>
<li>- <a href="#654166">What ruler should I use to measure the "uniqueness" of a song? Is that ruler selection justified?</a></li>
<li>- <a href="#515">Is there a complete database of hit music lyrics I can just download?</a></li>
<li>- Oh damn, that lyrics database doesn't exist. <a href="#733">How do I make it?</a></li>
<li>- <a href="#823">Is the data I have good?</a></li>
<li>- Ah man, the 5,700 song lyrics I've collected from four different sources have lots of errors in them - <a href="#1084">how do I clean them before becoming 72 years old?</a></li>
<li>- Ok, I thought I knew what words were but now I'm not sure. <a href="#1256">What is a word?</a> </li>
<li>- I've never made a website before - <a href="#12890">how do I make one to communicate my findings?</a></li>
</ul>
<p class="blurb">
... the other hundreds of questions mostly had to do with learning R (it was my first time), scraping websites (also my first time), and wrangling data.
Finally, one should ask if my findings align with other research. They do! The main conclusion from my work was that Rap music injected the Billboard 100 with a surprising amount of lyrical variety starting in the 90's... mainly from an explosion of artists using more slang in their prose. You'll observe that my last note in this journal was April 18th 2015 and two weeks later the Royal Society Open Science journal published a paper (see <a href="https://www.pbs.org/newshour/science/computer-scientists-prove-80s-music-boring">here</a> and <a href="http://rsos.royalsocietypublishing.org/content/2/5/150081">here</a>) in which the authors reached the same conclusion, albeit with far more fidelity (and expense). How exciting!?!</p>
<!-- <p class="blurb">2018 UPDATE: The New York Times published a neat piece <a href="https://www.nytimes.com/interactive/2018/08/09/opinion/do-songs-of-the-summer-sound-the-same.html">"Why Songs of the Summer Sound the Same"</a> based on acoustic analysis by music streaming service Spotify. In some ways the research validates my 2015 findings but maybe invalidates them in others. I'm still unsure. For example we agree that radio listeners in the late 80's and 90's enjoyed some of the highest variety music they had ever heard. Yet Spotify's analysis suggests that 2009 - 2012 saw a slump in uniqueness while my findings showed that the unique word count throughout the 2000's was about the same is in the 90's (which was between 75 and 350). Still yet, maybe Spotify's analysis and mine are saying the same things in different ways; After all, if the average "unique word count" of top songs hasn't changed in over 20 years then maybe they would seem more unique if that average where to just change modestly.. In the same way that a new outfit looses it's appeal if worn two weeks in a row.
</b></p> -->
<!--dygraph-->
<div>
<iframe width="100%" height="480" frameborder="0" seamless="seamless" scrolling="no" src="QA Graph.html" ></iframe>
</div>
<!--Notes-->
<div class="main">
<ul class="collapsible" data-collapsible="expandable">
<li>
<div id="654166" class="collapsible-header active"><span class="numbering">0</span>What definition of "Unique" will I use to measure song "Uniqueness?"<span class="cdates" style="float:right">2/10/2015 7:16 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>There are several ways one can define a measure of "uniqueness" for music; lyrical uniqueness, melodic uniqueness, idea or concept uniqueness (i.e. is a song about love or triumph). Perhaps a few more... The complete measure of a song's uniqueness is probably a combination of all of those things. But it's too expensive to develop instrumentation for measuring all of those. Instead, I will use "unique word's per song" as my uniqueness measure.
</div><div><br/></div><div>
This approach certainly has its flaws but it's probably not far from the truth. After all a chorus lyric tends to repeat in tandem with chorus melodics and chorus concepts / ideas (the lyric supplies the idea). So while crude, I'll assume that "unique word's per song" is a fair enough proxy for the thing I seek to measure here.
</div></div>
</div>
</div>
</li>
<li>
<div id="511" class="collapsible-header"><span class="numbering">1</span>Can multiple dygraphs [r] be stitched together in one browser window to make a sweet infographic?<span class="cdates" style="float:right">2/10/2015 7:30 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Theoretically yes - if dygraphs is creating javascript to render it's graphs (which it is) then I would think those same elements and canvas could be reformatted, in a copy/past&tweak fashion, to fit multiple dygraph canvases on one webpage.
</div>
</div>
</div>
</li>
<li>
<div id="513" class="collapsible-header"><span class="numbering">2</span>What doesn't a master want (in an apprentice)?<span class="cdates" style="float:right">2/10/2015 7:42 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Someone who...
<div><ul><li>... is going to quit</li><li>... doesn't have the ability to learn independently</li><li>... doesn't have a desire to know about the subject</li><li>... doesn't want to teach people (part of learning)</li><li>... is vague</li><li>... only tells them things they already know.</li><li>... will accomplish nothing with entrusted information - impart little impact.</li><li>... is a douchebag - nobody wants to work with a douchebag!</li></ul><div>My in...</div><div>If I can show a master that I am the opposite of the above I will almost certainly be allowed apprenticeship (accepted into grad school). </div><div><br/></div><div>UPDATE 1:</div><div>Maybe</div></div></div>
</div>
</div>
</li>
<li>
<div id="515" class="collapsible-header active"><span class="numbering">3</span>Where can I get lots of reliable music lyrics?<span class="cdates" style="float:right">2/11/2015 6:49 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Google provides some <a href="http://techcrunch.com/2014/12/22/google-adds-song-lyrics-to-top-of-search-results-points-searchers-to-google-play/">lyrics in search as of week of Dec 22 2014</a><div> LyricFind licenses lyrics to third parties like Pandora, SoundHound, Shazam and others.</div><div><br/></div><div>AZ Lyrics is like the Wikipedia for lyrics - crowd sourced (and open to vandalism?).</div><div><br/></div><div>The <a href="http://labrosa.ee.columbia.edu/millionsong/pages/additional-datasets">MillionSongDatabase</a> points to musiXmatch for lyrics.</div><div><br/></div><div>musiXmatch seems to source its lyric data from a combination of <a href="https://developer.musixmatch.com/faq">community and direct publisher agreements</a> but does not provide <b>full</b> lyric content for free or in the dataset produced for MillionSongDatabase.</div><div><br/></div><div>Conclusion:</div><div>No single source (known) offers free, full, direct from source (artist/publisher) lyrics. Perhaps the only way to obtain full lyric data is to scrape two or three lyric aggregation sites and have them crossed referenced/edited for accuracy and, most importantly, disallowed from public or private distribution.</div><div><br/></div><div>UPDATE1:</div><div>Or perhaps <a href="http://lyrics.wikia.com/Lyrics_Wiki">LyricWikia</a> is the Wikipedia of lyrics</div></div>
</div>
</div>
</li>
<li>
<div id="517" class="collapsible-header active"><span class="numbering">4</span>What is this project's scope?<span class="cdates" style="float:right">2/11/2015 6:54 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Desired: To analyze lyric and duration data for every predominantly english (as in <b>not</b> Gangnam Style) song on the billboard 100 since it's inception.
<div><br/></div><div>Stretch Goal Scope: To be determined.</div></div>
</div>
</div>
</li>
<li>
<div id="519" class="collapsible-header"><span class="numbering">5</span>When was the Billboard Hot 100 started?<span class="cdates" style="float:right">2/11/2015 6:56 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<a href="http://en.wikipedia.org/wiki/Billboard_Hot_100">August 4, 1958</a>
<div> As of the issue for the week ending February 21, 2015, the Hot 100 has had 1,041 different number-one hits.</div>
</div>
</div>
</div>
</li>
<li>
<div id="521" class="collapsible-header active"><span class="numbering">6</span>How does Billboard track who's listening to radio when?<span class="cdates" style="float:right">2/11/2015 7:05 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div><a href="http://en.wikipedia.org/wiki/Billboard_Hot_100">Word of mouth from DJ's, radio hosts and the like</a> coupled with hand filled surveys, and sales data from music stores and music networks formed 'popularity' indicators for a majority of Billboards existence. 'Popularity' sampling from consumer surveys by Arbitron (a consumer research company that became Nielsen Audio), evolved into wearable technologies such as the <a href="http://en.wikipedia.org/wiki/Nielsen_Audio">Portable People Meter</a> that could be given to consumers to wear for one to two years. People are paid to wear this little 'audio spy' for months while <a href="http://en.wikipedia.org/wiki/Nielsen_Broadcast_Data_Systems">it identifies what song, if any, you're listening to</a>.</div>
<div>
<br/>
</div>
<div>These audio spies are still used today along with audience metrics from digital sources such as Internet radio to enhance 'popularity' estimates.</div>
<div>
<br/>
</div>
<div>In summary, although Billboard's collection methodologies have changed many times since it's inception in 1958 and it's 'popularity' indexes have no doubt been biased to particular metrics at any given time (and indeed have failed to represent widely popular song from certain artists), its ability to fulfill the basic function of gauging a song's relative popularity is adequate for this project - it's a zeitgeist.</div>
</div>
</div>
</div>
</li>
<li>
<div id="523" class="collapsible-header active"><span class="numbering">7</span>Where can I find the name of every song ever on the Billboard Hot 100?<span class="cdates" style="float:right">2/11/2015 7:35 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Wikipedia:</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;">
<div><a href="http://en.wikipedia.org/wiki/Billboard_Hot_100">1,041 different number-one hits between Feb 2015 and Aug 1958</a> seems too few. </div>
<div>1041/57=19 songs a year or, spun another way, is like singing the same song for 2.5 weeks... On second thought - that sounds about right.</div>
</blockquote>
</div>
</div>
</div>
</li>
<li>
<div id="525" class="collapsible-header active"><span class="numbering">8</span>Can I use contact info of the last person to edit the Billboard Hot 100 wiki to find someone who can provide the songs list?<span class="cdates" style="float:right">2/11/2015 9:43 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Hello Ericorbit,<br/>
I happened upon the Billboard Hot 100 page you contribute to regularly and was wondering what the source is for your 'different number-one hits' figure. In your revision between '2015-01-01T16:54:54' and '2015-01-07T20:08:07' this figure changed from 1,040 to 1,041. Beyond manually (programmatically) counting entries in https://en.wikipedia.org/wiki/List_of_Billboard_number-one_singles how do you happen across this figure? Minor Googling doesn't reveal a straight answer. Help?<br/>
Thank you for your time ~~~~
<div><br/></div><div><br/>
PS: I'm asking because I'm putting together a song lyric database that I'll analyze with [R] to learn neat things about American music listenership. People who help me get early access to results in two months (yes - that's a music nerd bribe)</div></div>
</div>
</div>
</li>
<li>
<div id="527" class="collapsible-header active"><span class="numbering">9</span>How do I scrape wiki lists of historic billboard song ratings/titles?<span class="cdates" style="float:right">2/11/2015 10:32 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Year end ranks reported on Wikipedia are listed differently in 1958 from those between 1959 and 2014... 2015 will not be available till 2016.</div>
<div>
<br/>
</div>
<div>1958... although lists are available as early as 1956</div>
<div> <a href="https://en.wikipedia.org/wiki/Billboard_year-end_top_100_singles_of_1958">https://en.wikipedia.org/wiki/Billboard_year-end_top_100_singles_of_1958</a></div>
<div>
<br/>
</div>
<div>Remaining Wiki urls have the syntax:</div>
<div> <a href="https://en.wikipedia.org/wiki/Billboard_Year-End_Hot_100_singles_of_1959">https://en.wikipedia.org/wiki/Billboard_Year-End_Hot_100_singles_of_1959</a></div>
<div>where the article year ranges from 1959 to 2014</div>
<div>
<br/>
</div>
<div>
<a href="http://stackoverflow.com/questions/1395528/scraping-html-tables-into-r-data-frames-using-the-xml-package">Scraping html tables into R data frames using the XML package</a>
</div>
<div>install.packages("XML")</div>
<div>
<br/>
</div>
<div>The following [r] code gets most of the way...</div>
<div>
<br/>
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;">
<div>library(XML)</div>
<div>theurl <- "http://en.wikipedia.org/wiki/Billboard_year-end_top_100_singles_of_1958"</div>
<div>tables <- readHTMLTable(theurl)</div>
<div>write.csv(tables[[1]] [2:3], file = "Billboard Hot100 1958.csv", quote=FALSE)</div>
<div>
<br/>
</div>
<div>baseurl<-"http://en.wikipedia.org/wiki/Billboard_Year-End_Hot_100_singles_of_"</div>
<div>for (yr in 1959:2014 ) {</div>
<div>#Wikipedia markups such as "This article has multiple issues" will break this loop</div>
<div> stringyr <- as.character(yr)</div>
<div> theurl <- paste0(baseurl, stringyr)</div>
<div> tables <- readHTMLTable(theurl)</div>
<div> fileName <- paste0("Billboard Hot100 ", stringyr, ".csv")</div>
<div> write.csv(tables[[1]] [2:3], file = fileName, quote=FALSE)</div>
<div>}</div>
</blockquote>
</div>
</div>
</div>
</li>
<li>
<div id="529" class="collapsible-header"><span class="numbering">10</span>How long would it take to listen to all the Billboard songs?<span class="cdates" style="float:right">2/11/2015 10:46 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Figure 3 minute low average estimate, 4:30 high average estimate per song.
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;"><div>Low: 3:00*1041=3123 minutes -> 52 hours</div><div>High: 4.50*1041=4685 minutes -> 78 hours</div><div><br/></div></blockquote>
Between two and threeish days.
</div>
</div>
</div>
</li>
<li>
<div id="531" class="collapsible-header"><span class="numbering">11</span>Should song relationships be plotted as a function of 'Ranking date' or 'Song release date'?<span class="cdates" style="float:right">2/11/2015 11:04 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Popularity, after all, is temporal and time <i>does</i> matter. Since a goal of this study is to asses musical appetite (do Americans prefer broccoli or pizza... simple 'repetitive choruses' or <a href="https://en.wikipedia.org/wiki/Rap_God">complicated, dense verse</a>?) and the capacity for a song to become popular, ranking date will be sought firstly and release date as fall-back.
<div><br/></div><div><br/></div></div>
</div>
</div>
</li>
<li>
<div id="533" class="collapsible-header"><span class="numbering">12</span>How long does it take for a song to become ranked (popular)?<span class="cdates" style="float:right">2/11/2015 11:14 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Compare release date to rank date when data becomes available.
<div><br/></div></div>
</div>
</div>
</li>
<li>
<div id="535" class="collapsible-header"><span class="numbering">13</span>Is it worth it to subscribe to Billboard magazine to (more) easily get this data?<span class="cdates" style="float:right">2/11/2015 11:47 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Almost certainly. But I'd have to feed the machine that is Billboard. Then again, publishing <i>this</i> research might feed the machine.
<div><br/></div><div>ANSWER:</div><div>Unnecessary to subscribe to Billboard magazine.</div></div>
</div>
</div>
</li>
<li>
<div id="541" class="collapsible-header"><span class="numbering">14</span>How do you search reminders in Evernote?<span class="cdates" style="float:right">2/11/2015 11:57 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
remindertime:*
<div><br/></div><div>also use the reminder tab</div></div>
</div>
</div>
</li>
<li>
<div id="543" class="collapsible-header active"><span class="numbering">15</span>Why doesn't the "Scraping html tables into R data frames using the XML package" StackOverflow answer work?<span class="cdates" style="float:right">2/12/2015 12:29 AM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div><a href="http://stackoverflow.com/questions/1395528/scraping-html-tables-into-r-data-frames-using-the-xml-package">This answer</a> uses http in the example. Wikipedia will load in https depending on your browser, Wikipedia itself or who knows what. Removing the 's' from 'https' in the following code made it work</div>
<div>
<br/>
</div>
<div>theurl <- "https://en.wikipedia.org/wiki/Billboard_Year-End_Hot_100_singles_of_1959"<br/>
tables <- readHTMLTable(theurl)<br/>
n.rows <- unlist(lapply(tables, function(t) dim(t)[1]))</div>
<div>
<br/>
</div>
<div>Reminder:</div>
<div>Add this note to stackoverflow when I earn 50 reputation.</div>
<div>
<br/>
</div>
<div>
<br/>
</div>
</div>
</div>
</div>
</li>
<li>
<div id="545" class="collapsible-header"><span class="numbering">16</span>How do I extract a desired list among several in an [r] table?<span class="cdates" style="float:right">2/12/2015 12:43 AM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Try tables[[1]]</div>
<div>
<br/>
</div>
<div>As it turns out the answer was also on the <a href="http://stackoverflow.com/questions/1395528/scraping-html-tables-into-r-data-frames-using-the-xml-package">stackoverflow answer that recommended readHTMLTable()</a></div>
</div>
</div>
</div>
</li>
<li>
<div id="547" class="collapsible-header"><span class="numbering">17</span>How do you get table dimensions in [r]?<span class="cdates" style="float:right">2/12/2015 12:47 AM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>dim(x)<br/></div>
<div>
<br/>
</div>
<div>Examples</div>
<div>x <- 1:12 ; dim(x) <- c(3,4)<br/>
x<br/></div>
</div>
</div>
</div>
</li>
<li>
<div id="549" class="collapsible-header"><span class="numbering">18</span>What is an [R] table list type?<span class="cdates" style="float:right">2/12/2015 12:52 AM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>The type of a variable can be found using the <a href="http://stackoverflow.com/questions/12693908/r-get-type-of-variable">typeof()</a> function.</div>
<div>
<br/>
</div>
<div><a href="http://www.cyclismo.org/tutorial/R/types.html">A data frame is a way to take many vectors of different</a> types and store them in the same variable. The vectors can be of all different types. For example, a data frame may contain many lists, and each list might be a list of factors, strings, or numbers. </div>
<div>
<br/>
</div>
<div>
<br/>
</div>
</div>
</div>
</div>
</li>
<li>
<div id="551" class="collapsible-header"><span class="numbering">19</span>How do you read output from readHTMLTable [r]?<span class="cdates" style="float:right">2/12/2015 12:58 AM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<a href="http://web.mit.edu/~r/current/arch/i386_linux26/lib/R/library/XML/html/readHTMLTable.html">Read data from one or more HTML tables</a>
</div>
<div> tables downloaded in r by readHTMLTable() can be accessed by using double brackets i.e</div>
<div> tables = readHTMLTable(u)</div>
<div> tables[[1]]</div>
<div> </div>
<div>As it turns out the answer was also on the <a href="http://stackoverflow.com/questions/1395528/scraping-html-tables-into-r-data-frames-using-the-xml-package">stackoverflow answer that recommended readHTMLTable()</a></div>
</div>
</div>
</div>
</li>
<li>
<div id="553" class="collapsible-header"><span class="numbering">20</span>How do I save a table to file [r]?<span class="cdates" style="float:right">2/12/2015 1:05 AM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div><a href="http://stat.ethz.ch/R-manual/R-devel/library/utils/html/write.table.html">write.table</a></div><div><br/></div><div>There is also</div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;"><div>write.csv(x, file = "foo.csv")</div></blockquote>
and the inverse, reading, operation<br/><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;"><div>read.csv("foo.csv", row.names = 1)</div><div><a href="https://stat.ethz.ch/R-manual/R-devel/library/utils/html/read.table.html">read.csv(file_name, header = TRUE, sep = ",")</a></div><div><br/></div></blockquote>
Remove double quotes with quote=FALSE as in
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;"><div>write.csv(tables[[1]] [2:3], file = "foo.csv", quote=FALSE)</div></blockquote></div>
</div>
</div>
</li>
<li>
<div id="561" class="collapsible-header"><span class="numbering">21</span>Should double quotes introduced from wikipedia download be removed?<span class="cdates" style="float:right">2/12/2015 9:29 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Perhaps not - since double quotes are used to make exact Google searched. Wont add very much read time since population is small.
<div>... After additional deliberation - YES! Remove the crap out of them.</div><div><br/></div><div>UPDATE 1: Don't remove double quotes! Any string that contains a comma unenclosed by double quotes will erroneously add columns when written as a csv. DUHH </div></div>
</div>
</div>
</li>
<li>
<div id="563" class="collapsible-header"><span class="numbering">22</span>How should I handle/represent purely instrumental hits?<span class="cdates" style="float:right">2/12/2015 9:32 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Options:</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;">1) According to assumption "if it's Billboard ranked... it's popular enough" instrumental song counts should be reported but perhaps linked to a separate chart.
<div><br/></div><div>2) ...</div></blockquote>
</div>
</div>
</div>
</li>
<li>
<div id="565" class="collapsible-header"><span class="numbering">23</span>How do I make for loops in [r]?<span class="cdates" style="float:right">2/12/2015 9:38 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<a href="http://paleocave.sciencesortof.com/2013/03/writing-a-for-loop-in-r/">How to loop in R</a>
</div>
<div>
<br/>
</div>
<div>foo = seq(1, 100, by=2)</div>
<div>
<br/>
</div>
<div>foo.squared = NULL</div>
<div>
<br/>
</div>
<div>for (i in 1:50 ) {</div>
<div>foo.squared[i] = foo[i]^2</div>
<div>}</div>
</div>
</div>
</div>
</li>
<li>
<div id="567" class="collapsible-header"><span class="numbering">24</span>How are substrings manipulated in [r]?<span class="cdates" style="float:right">2/12/2015 9:44 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<a href="http://www3.nd.edu/~sjones20/JonesUND/BioStats_files/RstringManipulation_2-6-13.pdf">String manipulation in R</a>
</div>
<div>
<br/>
</div>
<div><a href="http://stat.ethz.ch/R-manual/R-devel/library/base/html/paste.html">Concatenate strings</a> using paste() to introduce spaces between concatenations and paste0() to concatenate without space.</div>
<div>
<br/>
</div>
<div>
<br/>
</div>
</div>
</div>
</div>
</li>
<li>
<div id="569" class="collapsible-header"><span class="numbering">25</span>How are numbers converted to strings in [r]?<span class="cdates" style="float:right">2/12/2015 9:50 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<code>
<a href="http://stackoverflow.com/questions/4104359/integer-to-string-conversion-in-r">as.character()</a>
</code>
</div>
</div>
</div>
</li>
<li>
<div id="573" class="collapsible-header active"><span class="numbering">26</span>What is the [r] equivalent to MATLAB 'try'?<span class="cdates" style="float:right">2/12/2015 10:55 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<a href="http://mazamascience.com/WorkingWithData/?p=912">Basic error handling with tryCatch()</a>
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;">
<div>result = tryCatch({</div>
<div> expr</div>
<div>}, warning = function(w) {</div>
<div> warning-handler-code</div>
<div>}, error = function(e) {</div>
<div> error-handler-code</div>
<div>}, finally = {</div>
<div> cleanup-code</div>
<div>}</div>
</blockquote>
</div>
</div>
</div>
</li>
<li>
<div id="575" class="collapsible-header"><span class="numbering">27</span>Is there some simple utility... wrapper I can use for [r] syntax highlighting on my website?<span class="cdates" style="float:right">2/12/2015 11:04 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Something like <a href="https://wordpress.org/plugins/crayon-syntax-highlighter/">Crayon Syntax Highlighter</a> but for js.</div>
</div>
</div>
</div>
</li>
<li>
<div id="577" class="collapsible-header"><span class="numbering">28</span>How do you count unique words in a string [r]?<span class="cdates" style="float:right">2/12/2015 11:14 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Try the command <a href="http://http//stackoverflow.com/questions/21621875/how-to-find-unique-words-given-a-list-of-words-in-r">unique</a> dummy</div>
<div>Also get into the habit of first searching for command help in [r] by typing ?'the-command' or ??'the-command' ... just like in MATLAB.</div>
<div>
<br/>
</div>
<div>
<a href="http://stackoverflow.com/questions/28033312/r-how-do-keep-only-unique-words-within-each-string-in-a-vector">R How do keep only unique words within each string in a vector</a>
</div>
</div>
</div>
</div>
</li>
<li>
<div id="581" class="collapsible-header active"><span class="numbering">29</span>What types of questions could this project's unstructured data answer?<span class="cdates" style="float:right">2/13/2015 4:49 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Maintaining that the extent of 'unstructured' data I will harvest are lyrics, perhaps it would be cool to distill emotion/theme/genre categorizations from songs... That is... to answer does American popular opinion prefer...
<div><ol><li>Love songs over breakup songs?</li><li>Triumph songs over anger songs?</li><li>Selfish over selfless songs?</li><li><br/></li></ol></div></div>
</div>
</div>
</li>
<li>
<div id="585" class="collapsible-header"><span class="numbering">30</span>How to scrape Google of lyrics?<span class="cdates" style="float:right">2/13/2015 10:47 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>According to <a href="http://techcrunch.com/2014/12/22/google-adds-song-lyrics-to-top-of-search-results-points-searchers-to-google-play/">TechCrunch</a> song lyrics are now being provided at the top of Google search results pages.</div>
<div>
<br/>
</div>
<div>UPDATE 1:</div>
<div>These can be scraped in part using techniques described in <a href="#672" style="color: rgb(105, 170, 53);" data-click="true">What is a simple RCurl example I can use to begin learning how to use RCurl?</a></div>
</div>
</div>
</div>
</li>
<li>
<div id="587" class="collapsible-header"><span class="numbering">31</span>What portion of historic Billboard Hot 100 songs does the Million Song Dataset contain?<span class="cdates" style="float:right">2/13/2015 10:59 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
It appears neither the Million Song Dataset nor the musiXmatch data set it also references offer full lyric data since doing so is copyright blocked... so this question doesn't matter.
</div>
</div>
</div>
</li>
<li>
<div id="589" class="collapsible-header"><span class="numbering">32</span>How do you make 3D graphs in [r]?<span class="cdates" style="float:right">2/14/2015 3:51 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div><a href="http://cran.r-project.org/web/packages/plot3D/vignettes/plot3D.pdf">plot3D</a><a href="http://cran.r-project.org/web/packages/plot3D/vignettes/plot3D.pdf">: Tools for plotting 3-D and 2-D data</a></div><div><br/></div><div><a href="http://blog.revolutionanalytics.com/2014/02/3d-plots-in-r.html">3D Plots in R - Revolution Analytics</a></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><a href="http://cran.r-project.org/web/packages/scatterplot3d/vignettes/s3d.pdf">Scatterplot3d –</a><a href="http://cran.r-project.org/web/packages/scatterplot3d/vignettes/s3d.pdf">an R </a><a href="http://cran.r-project.org/web/packages/scatterplot3d/vignettes/s3d.pdf">package for Visualizing Multivariate Data</a></div><div><a href="http://pj.freefaculty.org/guides/Rcourse/plot-3d/plots-3d.pdf">3D Plotting - </a><a href="http://pj.freefaculty.org/guides/Rcourse/plot-3d/plots-3d.pdf">Paul E. Johnson's</a></div><div><br/></div></blockquote><a href="https://plot.ly/r/3d-surface-plots/">Plot.ly looks promising</a> - it has an elegance similar to dygraph.
</div>
</div>
</div>
</li>
<li>
<div id="591" class="collapsible-header"><span class="numbering">33</span>Did I overlook wikipedia's total Billboard Hot 100's total song count reference?<span class="cdates" style="float:right">2/14/2015 4:43 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
In reference to 'When was the Billboard Hot 100 started?', as far as I can tell, the total song count to date of 1041 IS unreferenced.
</div>
</div>
</div>
</li>
<li>
<div id="597" class="collapsible-header"><span class="numbering">34</span>What is Plot.ly?<span class="cdates" style="float:right">2/14/2015 4:54 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
It's awesome - an online plotting tool with libraries for plotting in popular technical tools such as [r], MATLAB and Python to enhance, share and backup plots.
<div> <a href="http://en.wikipedia.org/wiki/Plotly">http://en.wikipedia.org/wiki/Plotly</a></div><div><br/></div><div><a href="https://plot.ly/streaming/">You can stream to it!</a></div></div>
</div>
</div>
</li>
<li>
<div id="599" class="collapsible-header active"><span class="numbering">35</span>Where did this guy get his data from and how is my work different from his?<span class="cdates" style="float:right">2/14/2015 5:04 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>While looking into the 3D capabilities of Plot.ly I stumbled upon this graph of <a href="https://plot.ly/~RhettAllain/131/average-song-length/">Average Song Length since 1945ish</a>. I have a problem with this chart: the data source is a complete mystery. I have no clear way of deducing what songs were considered.</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;">
<div>
<br/>
</div>
<div>Apparently he, <a href="https://plot.ly/~RhettAllain">RhettAllain</a>, writes for Wired Science. </div>
<div>
<br/>
</div>
<div>I like his error bars.</div>
</blockquote>
<div>
<br/>
</div>
<div>Forget about pursuing his source... how will my work be different?</div>
<div>
<ul>
<li>I'll be more transparent</li>
<li>I'll attempt to bin averages across more relevant time intervals (annual seems as abstract an interval as 213 days - is one better than the other?)</li>
<li>3D Plot</li>
</ul>
<div>UPDATE 1:</div>
</div>
<div>Rhett Allain produced his graph in relation to the WIRED article he wrote - <a href="http://www.wired.com/2014/07/why-are-songs-on-the-radio-about-the-same-length/">Why Are Songs on the Radio About the Same Length?</a></div>
</div>
</div>
</div>
</li>
<li>
<div id="601" class="collapsible-header"><span class="numbering">36</span>How do I get Evernote to default the cursor of a new note to the Title field?<span class="cdates" style="float:right">2/14/2015 5:23 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><a href="https://discussion.evernote.com/topic/41596-feature-request-change-cursor-default-setting-to-note-title-field/">Apparently this option is not configurable</a>. However, one can press F2 on a note to switch focus to the Title field. Alternatively, the following AutoHotkey code can bind the creation of a new note to the keyboard sending of the 'F2' key all in one go (tested/works).
<div><pre>
#IfWinActive, ahk_class ENMainFrame
^n::
SendInput ^n
Sleep 100
SendInput {F2}
Return
</pre></div></div>
</div>
</div>
</li>
<li>
<div id="603" class="collapsible-header active"><span class="numbering">37</span>What kind of data is available in the millions song Dataset?<span class="cdates" style="float:right">2/14/2015 7:50 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><a href="http://labrosa.ee.columbia.edu/millionsong/pages/field-list">A lot of tune and harmony type data</a> as well as metadata like artist name, song duration and year.
</div>
</div>
</div>
</li>
<li>
<div id="607" class="collapsible-header"><span class="numbering">38</span>What is the Echo Nest?<span class="cdates" style="float:right">2/14/2015 8:25 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
A "music intelligence platform [that] <a href="http://the.echonest.com/">synthesizes billions of data points</a> and transforms it into musical understanding..."
<div> Engadget has noted they are "The song-picking puppet master pulling the playlist strings behind iHeartRadio, Spotify and Nokia's music services."</div></div>
</div>
</div>
</li>
<li>
<div id="609" class="collapsible-header"><span class="numbering">39</span>Uhhh... What?<span class="cdates" style="float:right">2/14/2015 8:32 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<a href="http://youtu.be/SDpC5ZYcA7M?t=22s">CataCombo Sound System by Pause</a>
</div>
</div>
</div>
</div>
</li>
<li>
<div id="611" class="collapsible-header"><span class="numbering">40</span>What are stemmed/unstemmed words?<span class="cdates" style="float:right">2/14/2015 9:23 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>The goal of stemming is that many related words are mapped onto the same one. For instance <a href="http://labrosa.ee.columbia.edu/millionsong/musixmatch">'victori' maps to 'victory'</a> and ought be counted as the same word.</div>
<div>
<br/>
</div>
<div>UPDATE 1:</div>
<div>The following SO post might help mapping endeavors <a href="http://stackoverflow.com/questions/6954017/r-replace-characters-using-gsub-how-to-create-a-function">R: replace characters using gsub, how to create a function?</a></div>
<div>Perhaps <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/chartr.html">chartr()</a> is of use - Translate characters in character vectors, in particular from upper to lower case or vice versa.</div>
</div>
</div>
</div>
</li>
<li>
<div id="613" class="collapsible-header"><span class="numbering">41</span>What are musiXmatch's Terms of Use?<span class="cdates" style="float:right">2/14/2015 9:54 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<a href="https://about.musixmatch.com/eula/">EULA found here</a>
<div>
<br/>
</div>
<div>Important points:</div>
<div>6 You will not copy any part of Musixmatch or any Third Party Applications and/or Sites or make commercial use of, rent, lease, loan, sell, publish, license, sublicense, distribute, assign or otherwise transfer any part of Musixmatch to any person.</div>
<div>
<br/>
</div>
<div>6.6 Any content provided by us as part of Musixmatch, including but not limited to ringtones, lyrics, artist information and downloads contains copyrighted material, trademarks and other proprietary rights belonging to us and our licensors. All right, title and interest in and to such content vests in us and our licensors. You are granted a limited, revocable, non-exclusive licence to display that content as part of the Services solely for your personal use. Except as expressly authorized by us, you may not copy, modify, translate, reproduce, distribute, publish, broadcast, perform, display, sell, assign, lease or sub-license that content, in whole or in part. </div>
<div>
<br/>
</div>
<div>
<a href="https://about.musixmatch.com/copyright/">Copyright found here</a>
</div>
<div>
<br/>
</div>
<div>
<br/>
</div>
</div>
</div>
</div>
</li>
<li>
<div id="615" class="collapsible-header"><span class="numbering">42</span>What are AZLyrics Terms of Use?<span class="cdates" style="float:right">2/14/2015 10:02 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<a href="http://www.azlyrics.com/copyright.html">Copyright found here</a>
</div>
<div>Important points:</div>
<div>Nothing notable, however, AZ Lyrics is 'powered' by musixmatch.</div>
<div>
<br/>
</div>
<div>In reference to <a href="#513" data-click="true">What doesn't a master want (in an apprentice)?</a> people (in general) also tend not to like to work with criminals... respect the Digital Millennium Copyright Act.</div>
</div>
</div>
</div>
</li>
<li>
<div id="617" class="collapsible-header"><span class="numbering">43</span>Does the scraped Wikipedia Billboard Hot 100 data have any duplicates?<span class="cdates" style="float:right">2/14/2015 10:17 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<a href="#823" data-click="true">Yes</a>
</div>
</div>
</div>
</li>
<li>
<div id="619" class="collapsible-header"><span class="numbering">44</span>How do you list a size summary of table of lists in [r]?<span class="cdates" style="float:right">2/14/2015 10:23 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Use summary()</div>
<div>
<br/>
</div>
<div>Sometimes [r] is just plain easy. Then again... everything is easy once you know how to do it. </div>
</div>
</div>
</div>
</li>
<li>
<div id="625" class="collapsible-header"><span class="numbering">45</span>Does Ryan Tedder mostly only write lyrics?<span class="cdates" style="float:right">2/14/2015 10:48 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Ryan Tedder has written a <a href="http://en.wikipedia.org/wiki/Ryan_Tedder#Producing_and_writing">ton of hit songs</a>. </div>
</div>
</div>
</div>
</li>
<li>
<div id="627" class="collapsible-header"><span class="numbering">46</span>Who has written (or co-written) the highest number of Billboard Hot 100 songs?<span class="cdates" style="float:right">2/14/2015 11:15 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Let the numbers speak. To be answered later.
</div>
</div>
</div>
</li>
<li>
<div id="631" class="collapsible-header"><span class="numbering">47</span>What exactly does getURL in [r] get?<span class="cdates" style="float:right">2/14/2015 11:23 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
The getURL and getURLContent functions from the RCurl package are used to <a href="http://rfunction.com/archives/1672">retrieve the source of a webpage</a>.
<div><br/></div><div><br/></div></div>
</div>
</div>
</li>
<li>
<div id="635" class="collapsible-header"><span class="numbering">48</span>How to get length of lists in list [r]?<span class="cdates" style="float:right">2/15/2015 12:24 AM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
This question was asked in hopes of answering another question - how do I only download the billboard tables from Wikipedia using
<div> tables <- readHTMLTable(theurl) </div><div><br/></div><div>As it turns out the longest list (who's length is 3) returned by the list of lists 'tables' is the list of interest. Use</div><div> tables[sapply(tables, length) == 3]</div><div>to return the desired ranking information. <a href="http://stackoverflow.com/questions/17821698/remove-list-from-lists-in-list-if-length">Recommended from StackOverflow</a>.</div></div>
</div>
</div>
</li>
<li>
<div id="639" class="collapsible-header"><span class="numbering">49</span>What is the most common Billboard Hot 100 phrase?<span class="cdates" style="float:right">2/17/2015 1:13 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>To be determined</div>
</div>
</div>
</div>
</li>
<li>
<div id="640" class="collapsible-header"><span class="numbering">50</span>What's more powerful: Lyrics or Tune?<span class="cdates" style="float:right">2/17/2015 1:15 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
To be determined
</div>
</div>
</div>
</li>
<li>
<div id="641" class="collapsible-header"><span class="numbering">51</span>What would a sentiment analysis of the Billboard Hot 100 say about american emotion?<span class="cdates" style="float:right">2/17/2015 1:13 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Keep this question for later</div>
<div>
<br/>
</div>
</div>
</div>
</div>
</li>
<li>
<div id="648" class="collapsible-header"><span class="numbering">52</span>How do you save table without row names [r]?<span class="cdates" style="float:right">2/21/2015 12:32 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
use <a href="http://stat.ethz.ch/R-manual/R-devel/library/utils/html/write.table.html">write.table</a><div> row.names = FALSE</div></div>
</div>
</div>
</li>
<li>
<div id="650" class="collapsible-header"><span class="numbering">53</span>How do you append multiple CSV files together [r]?<span class="cdates" style="float:right">2/21/2015 12:54 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
There are a ton of ways of doing this. This <a href="http://r.789695.n4.nabble.com/Import-Multiple-csv-files-and-merge-into-one-Master-file-td2967823.html">forum post</a> lists a couple.
<div><br/></div><div>Since my analysis will mix and match these rows (songs) at some point I ought to append year information.</div></div>
</div>
</div>
</li>
<li>
<div id="652" class="collapsible-header"><span class="numbering">54</span>How do you make a simple column vector with name in [r]?<span class="cdates" style="float:right">2/21/2015 1:05 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
A data frame is used for storing data tables. It is a list of vectors of equal length. For example, the following variable df is a <a href="http://www.r-tutor.com/r-introduction/data-frame">data frame containing three vectors</a> n, s, b.
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;"><div>n = c(2, 3, 5) </div><div>s = c("aa", "bb", "cc") </div><div>b = c(TRUE, FALSE, TRUE) </div><div>df = data.frame(n, s, b) # df is a data frame </div></blockquote></div>
</div>
</div>
</li>
<li>
<div id="656" class="collapsible-header"><span class="numbering">55</span>How to make a vector of all the same number [r]?<span class="cdates" style="float:right">2/21/2015 1:24 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<a href="http://stackoverflow.com/questions/3672527/r-generate-a-repeating-sequence-based-on-vector">R: generate a repeating sequence based on vector</a>
<div> rep(1995,20)</div>
<div> Output > [1] 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995 1995</div>
</div>
</div>
</div>
</li>
<li>
<div id="660" class="collapsible-header active"><span class="numbering">56</span>Do [r] matrices only support numbers?<span class="cdates" style="float:right">2/21/2015 1:48 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
No, they support characters or numbers and perhaps another type but <a href="http://stackoverflow.com/questions/13858471/how-to-change-matrix-column-type-in-r">only one type at a time</a>. To mix various data types use data.frame
<div><br/></div></div>
</div>
</div>
</li>
<li>
<div id="664" class="collapsible-header"><span class="numbering">57</span>How do I remove double quotation marks from data.frame [r]?<span class="cdates" style="float:right">2/21/2015 1:52 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Various online message boards for this question entertain answers that have to do with how [r] makes output more human readable by including imaginary quotation marks.
<div><br/></div><div>This is the <a href="http://stackoverflow.com/questions/21658113/remove-all-quotation-marks-from-a-data-frame">one answer</a> I could get to <i>actually</i> remove quotation marks in my data set. Note that it is not marked as <i>the</i> answer by the asker.</div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="Monospac821 BT">library(stringr)</font></div><div><span style="font-family: 'Monospac821 BT';">library(plyr)</span></div><div><span style="font-family: 'Monospac821 BT';">del <- colwise(function(x) str_replace_all(x, '\"', ""))</span></div><div><span style="font-family: 'Monospac821 BT';">x <- del(x)</span></div><div><font face="Monospac821 BT"><br/></font></div></blockquote><div>Return to this question to add comment when get SO 50 Rep.</div></div>
</div>
</div>
</li>
<li>
<div id="672" class="collapsible-header"><span class="numbering">58</span>What is a simple RCurl example I can use to begin learning how to use RCurl?<span class="cdates" style="float:right">2/21/2015 8:42 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>This example from <a href="http://rfunction.com/archives/1672">R Function ofthe Day</a></div>
<div>
<br/>
</div>
<div>URL <- "http://www.ebay.com/sch/ctg/Big-Bang-Theory-Complete-Fourth-Season-DVD-2011-3-Disc-Set-/103149230?LH_Auction=1&_dmpt=US_DVD_HD_DVD_Blu_ray&_pcategid=617&_pcatid=1&_refkw=big+bang+theory+season+4&_trkparms=65%253A12%257C66%253A4%257C39%253A1%257C72%253A5841&_trksid=p3286.c0.m14"</div>
</div>
</div>
</div>
</li>
<li>
<div id="674" class="collapsible-header"><span class="numbering">59</span>How do I know if I have a particular library installed in [r]?<span class="cdates" style="float:right">2/21/2015 9:28 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<a href="http://www.ats.ucla.edu/stat/r/faq/packages.htm">installed.packages()</a>
<div>
<br/>
</div>
<div>Check for a specific package:</div>
<div>
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;">
<div>
<pre>
<b>a<-installed.packages()
packages<-a[,1]
is.element("boot", packages)</b>
</pre>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</li>
<li>
<div id="680" class="collapsible-header"><span class="numbering">60</span>Where can I find RCurl documentation?<span class="cdates" style="float:right">2/24/2015 9:33 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<div>Notes from <a href="http://www.omegahat.org/RCurl/RCurlJSS.pdf">R as a Web Client – the </a><a href="http://www.omegahat.org/RCurl/RCurlJSS.pdf">RCurl </a><a href="http://www.omegahat.org/RCurl/RCurlJSS.pdf">package</a></div>
<blockquote style="margin: 0px 0px 0px 40px; border: none; padding: 0px;">
<div>Can be used to make concurrent webpage requests</div>
<div>Can process web pages in pieces [chunks] as they become available keeping memory overhead low</div>
<div>Example 5.1 doesn't work on account of the link <a href="http://www.omegahat.org/RCurl/exampleStock.dat">http://www.omegahat.org/RCurl/exampleStock.dat</a> no longer exists.</div>
<div>Appears to be written sometime in 2006 or 2007</div>
</blockquote>
<div>
<br/>
</div>
<div>
<a href="http://cran.r-project.org/web/packages/RCurl/index.html">Official Documentation</a>
</div>
</div>
</div>
</div>
</div>
</li>
<li>
<div id="682" class="collapsible-header"><span class="numbering">61</span>What's the simplest website that reports my IP?<span class="cdates" style="float:right">2/24/2015 9:56 PM</span></div>
<div class="collapsible-body">
<div class="note">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>This one told me the wrong IP... - <a href="http://www.whatsmyip.org/">http://www.whatsmyip.org/</a></div>
<div>I cannot find my ip in the page source code of:</div>
<div>
<a href="https://www.whatismyip.com/">https://www.whatismyip.com/</a>
</div>
<div>
<a href="http://whatismyipaddress.com/">http://whatismyipaddress.com/</a>
</div>
<div>
<br/>
</div>
<div>UPDATE!!</div>