[escape hatch: for those who care about manga, not process]
[blue portal: tl;dr – skip the boring bits AND the long lists and get to the final commentary]
So I had to go back 4 months to find my last couple of written columns about manga – and a lot has changed with my homebrew charts since then. The difficulty in keeping up with manga sales data has *always* been a matter of time: as the data is already out there. Anyone can visit Amazon & look at their hourly bestsellers at any time (or even multiple times a day). But to compile all the data from available sources was a rather painstaking process, especially since one cannot, for example, cut-n-paste the Amazon page into a spreadsheet to compile a composite chart. None of the sales sites use the same format or nomenclature, and it becomes not only a math exercise but also a translation problem; not comparing apples to oranges, but apples to softballs to aerogel to a cup of nonnewtonian fluid.
Most problems can be solved with the application of time. Compiling the manga bestsellers used to be very hands-on: overcoming differences in formats & listings with a lot of human-processing. By using _an actual person_ to read, analyze, & re-input the data from book sales sites into a single consistent format that could then be used in a spreadsheet, we bridge the machine-readable-to-human-readable gap: but the process is labor intensive. My small concern only has the one employee (me) and this same employee has to also hold down a 45hr. a week job (to pay the bills, as the data-analysis-stuff is still just a hobby). There weren’t enough hours in the week.
To compile one week’s worth of bestsellers I was spending 16 hours a week or more just clicking on websites & typing in data. It was a ‘brute force’ method: very easy to implement but not very efficient. It was also very easy to fall behind; I could always save pages for later data collection, but the actual data collection required me to sit in a chair, look at long lists, and type.
I started out a couple of weeks behind, but soon was starting at a backlog of two-and-a-half-month’s-worth of archived data and frankly, seeing no way out. I was tempted to just chuck the whole project, like I once did in 2009. I decided to keep at it though, thankless a task as this is, because no one else is tracking this data.
So obviously I needed a new approach. I had to teach myself a few new tricks.
##
Data collection for the manga charts is now handled by a set of programs — the technology is similar to web spiders or other big sites that seek to index the web (think Google) but my application is much, much smaller in scale. You could think of them as a type of ‘bot, programmed to love manga, crawl a very specific set of URLs and download just the rank & title of the manga volumes found there. I’ll admit to becoming fond of them, I call them my DRDs. Instead of manually loading websites & reading them myself (or archiving them for later) I just let the DRDs crawl the web for me and stash the results in a CSV file.
Problem One solved – but it introduces the second hurdle: One site will list Naruto 54, another Naruto vol. 54, and a third Naruto Volume 54 – and these are all the same book. You can see that, I can see that, but computers are a little particular and can’t make the connection right away. With my automatically collected raw data, I needed to then set up a filter to ‘translate’ each of the three listings into a single consistent format.
Luckily, I had a database of manga titles in my own indosyncratic format (compiled over the past 5 years) so it was merely a matter of teaching a spreadsheet to read the three Naruto 54s the same way. Merely. My kludge is yet another brute-force solution, though I’m letting my CPU be the brute in this case; my ‘filter’ takes the raw data provided by the DRDs and compares them to a set of 21558 definitions (as of 12 August 2012, and growing each week) to output a single set of data, while simultaneously kicking out the “non manga” [as defined] and returning a list, sorted by source & rank, and reformatted to my preferred title notation.
It sounds fancy, but it really just builds on the manga charts themselves: I’ve been teaching spreadsheets to compare-and-count “like” titles for years now, this is just a specific application. Still (as I am now finding in other hobby projects) this type of filter really does rely on a foundation of past data; this is a very hard process to recreate from scratch.
Source data is then weighted, scored, compiled, and run through the usual meat-grinder of a spreadsheet I’ve always used to output the charts. I’ve used this spreadsheet & format for a couple of years now; these are the charts that post weekly. This isn’t the part that’s new. What’s new is the new automated data collection process that pulls data from websites while I sleep & enables me to go from raw data to posted chart [with a couple rounds of manual error checking & correction] in under two hours. If you follow me on twitter you got to read my comments in real time while I worked on the charts just posted for 12 August.
Your takeaway from all the boring posted above is that it is now possible for my small datashop to process a whole week’s worth of online sales data and output the chart in a single day — even same day delivery, if I happen to get to it either before or after working at the bookstore.
But enough about process, let’s look at manga:
Wow. Months have passed since I last wrote a Manga Radar column, so I’m not even sure where to start. Obviously, I’ve added hundreds of title to the database in the intervening time, so a listing of just “what’s new” would be an overload. I’ll assume you know about new releases this month, and next (and I post each as it’s own chart so there are links for that) – so true to concept, let’s take a look at some of the titles fans are preordering that are even further out:
[the rankings listed are for the most recent chart posted, 12 August]
October Releases
2239. (new) : Angelic Layer Omnibus – Dark Horse, Oct 2012 [0.1] ::
37. ↑2 (39) : Avatar: The Last Airbender The Promise 3 – Dark Horse, Oct 2012 [188.0] ::
278. ↑9 (287) : Berserk 36 – Dark Horse, Oct 2012 [44.9] ::
234. ↑12 (246) : Cardcaptor Sakura Omnibus 4 – Dark Horse, Oct 2012 [51.9] ::
1796. ↓-510 (1286) : Bad Teacher’s Equation 5 – DMP Juné, Oct 2012 [0.1] ::
1782. ↓-4 (1778) : Honey*Smile – DMP Juné, Oct 2012 [0.1] ::
. ↑2536 (2536) : Secret Thorns – DMP Juné, Oct 2012 [0.0] ::
. (last ranked 10 Jun 12) : Love Hair – DMP Project H, Oct 2012 [0.0] ::
. (last ranked 17 Jun 12) : Love Infusion – DMP Project H, Oct 2012 [0.0] ::
240. ↑374 (614) : My Good Boy – DMP Project H, Oct 2012 [50.7] ::
248. ↑214 (462) : With A Dictionary & No Skirt – DMP Project H, Oct 2012 [49.6] ::
<
strong>274. ↓-23 (251) : Arisa 9 – Kodansha Comics, Oct 2012 [45.5] ::
. : Cage of Eden 8 – Kodansha Comics, Oct 2012 [0.0] ::
. : Kitchen Princess Omnibus 2 – Kodansha Comics, Oct 2012 [0.0] ::
164. ↑53 (217) : Mardock Scramble 6 – Kodansha Comics, Oct 2012 [70.0] ::
69. ↑3 (72) : Negima! 36 – Kodansha Comics, Oct 2012 [126.1] ::
342. ↓-33 (309) : Dance in the Vampire Bund 13 – Seven Seas, Oct 2012 [35.9] ::
157. ↑13 (170) : Girl Friends: Complete Collection 1 – Seven Seas, Oct 2012 [72.6] ::
270. ↓-26 (244) : Venus Versus Virus Omnibus 3 – Seven Seas, Oct 2012 [46.0] ::
1753. ↓-32 (1721) : Awkward Silence 2 – SuBLime, Oct 2012 [0.1] ::
. (last ranked 5 Aug 12) : His Favorite 2 – SuBLime, Oct 2012 [0.0] ::
. (last ranked 29 Apr 12) : Punch Up! 2 – SuBLime, Oct 2012 [0.0] ::
. : Flowers of Evil 3 – Vertical, Oct 2012 [0.0] ::
. (last ranked 5 Aug 12) : GTO The Early Years: Shonan Junai Gumi 15 – Vertical, Oct 2012 [0.0] ::
1675. ↓-3 (1672) : Loveless vols 1-2 collection – Viz, Oct 2012 [0.1] ::
2044. (new) : X vols 10-12 collection – Viz, Oct 2012 [0.1] ::
2247. (new) : Dawn of the Arcana 6 – Viz Shojo Beat, Oct 2012 [0.1] ::
. : Jiu Jiu 2 – Viz Shojo Beat, Oct 2012 [0.0] ::
341. ↑58 (399) : Skip Beat! 29 – Viz Shojo Beat, Oct 2012 [35.9] ::
1125. ↓-42 (1083) : Stepping on Roses 8 – Viz Shojo Beat, Oct 2012 [2.9] ::
1555. ↑41 (1596) : Story of Saiunkoku 8 – Viz Shojo Beat, Oct 2012 [0.2] ::
1321. ↑433 (1754) : Bakuman 15 – Viz Shonen Jump, Oct 2012 [1.2] ::
1043. ↑117 (1160) : Bleach 48 – Viz Shonen Jump, Oct 2012 [3.8] ::
996. ↓-28 (968) : Bleach 49 – Viz Shonen Jump, Oct 2012 [4.5] ::
2163. (last ranked 15 Apr 12) : Nura: Rise of the Yokai Clan 11 – Viz Shonen Jump, Oct 2012 [0.1] ::
766. ↓-77 (689) : Case Closed 44 – Viz Shonen Sunday, Oct 2012 [9.6] ::
2230. ↓-331 (1899) : Tenjo Tenge: Full Contact Edition 9 – Viz Signature, Oct 2012 [0.1] ::
721. ↑191 (912) : Pokemon Black & White vols 1-8 box set – Vizkids, Oct 2012 [11.3] ::
1379. ↓-58 (1321) : Pokemon Diamond & Pearl/Platinum 6 – Vizkids, Oct 2012 [0.8] ::
325. ↑78 (403) : Black Butler 11 – Yen Press, Oct 2012 [38.2] ::
751. ↑186 (937) : Durarara!! 4 – Yen Press, Oct 2012 [10.3] ::
1076. ↑106 (1182) : Haruhi Suzumiya The Disappearance of Nagato Yuki-Chan 2 – Yen Press, Oct 2012 [3.5] ::
552. ↑155 (707) : Pandora Hearts 12 – Yen Press, Oct 2012 [19.1] ::
1351. ↑853 (2204) : Soul Eater NOT! 2 – Yen Press, Oct 2012 [0.9] ::
. ↑ (last ranked 29 Jul 12) : Triage X 1 – Yen Press, Oct 2012 [0.0] ::
November Releases
1833. ↑246 (2079) : Bride of the Water God 12 – Dark Horse, Nov 2012 [0.1] ::
833. ↑124 (957) : Deva Zan – Dark Horse, Nov 2012 [8.0] ::
. (last ranked 29 Jul 12) : Gantz 25 – Dark Horse, Nov 2012 [0.0] ::
119. ↑99 (218) : Velvet Kiss 2 – DMP Project H, Nov 2012 [89.8] ::
1673. ↓-238 (1435) : Fairy Tail 22 – Kodansha Comics, Nov 2012 [0.1] ::
338. ↑1003 (1341) : Missions of Love 1 – Kodansha Comics, Nov 2012 [36.5] ::
303. ↓-40 (263) : Phoenix Wright Miles Edgeworth 3 – Kodansha Comics, Nov 2012 [40.9] ::
21. ↓-2 (19) : Sailor Moon 8 – Kodansha Comics, Nov 2012 [254.4] ::
666. ↑86 (752) : Alice in the Country of Clover Cheshire Cat Waltz 3 – Seven Seas, Nov 2012 [14.1] ::
2485. ↓-615 (1870) : Angel Para Bellum 2 – Seven Seas, Nov 2012 [0.0] ::
680. ↑124 (804) : Haganai: I Don’t Have Many Friends 1 – Seven Seas, Nov 2012 [13.3] ::
. (last ranked 11 Mar 12) : Young Miss Holmes vols 3-4 collection – Seven Seas, Nov 2012 [0.0] ::
1906. ↓-130 (1776) : Bond of Dreams, Bond of Love 2 – SuBLime, Nov 2012 [0.1] ::
1697. ↓-28 (1669) : Starting with a Kiss 2 – SuBLime, Nov 2012 [0.1] ::
. : GTO: 14 Days in Shonan 6 – Vertical, Nov 2012 [0.0] ::
1200. ↓-29 (1171) : Message to Adolf 2 – Vertical, Nov 2012 [2.2] ::
. (last ranked 13 May 12) : Excel Saga 24 – Viz, Nov 2012 [0.0] ::
1186. ↓-93 (1093) : Neon Genesis Evangelion 13 – Viz, Nov 2012 [2.3] ::
104. ↓-15 (89) : Nausicaa of the Valley of the Wind 2 vol hardcover box set [complete] – Viz Ghibli Library, Nov 2012 [100.1] ::
2486. (new) : Ai Ore! 7 – Viz Shojo Beat, Nov 2012 [0.0] ::
1685. ↑8 (1693) : Kamisama Kiss 11 – Viz Shojo Beat, Nov 2012 [0.1] ::
979. ↑97 (1076) : Kimi ni Todoke 15 – Viz Shojo Beat, Nov 2012 [4.8] ::
855. ↓-17 (838) : Ouran High School Host Club vols 1-18 box set – Viz Shojo Beat, Nov 2012 [7.2] ::
215. ↑75 (290) : Vampire Knight 15 – Viz Shojo Beat, Nov 2012 [57.4] ::
2256. (new) : Bakuman 16 – Viz Shonen Jump, Nov 2012 [0.1] ::
1829. ↓-10 (1819) : Bleach 50 – Viz Shonen Jump, Nov 2012 [0.1] ::
1866. ↓-64 (1802) : Bleach 51 – Viz Shonen Jump, Nov 2012 [0.1] ::
318. ↓-22 (296) : Naruto 59 – Viz Shonen Jump, Nov 2012 [39.1] ::
881. ↑482 (1363) : One Piece 65 – Viz Shonen Jump, Nov 2012 [6.7] ::
. : Psyren 7 – Viz Shonen Jump, Nov 2012 [0.0] ::
554. ↑50 (604) : Blue Exorcist 8 – Viz Shonen Jump Advanced, Nov 2012 [18.8] ::
626. ↓-17 (609) : Claymore 21 – Viz Shonen Jump Advanced, Nov 2012 [15.6] ::
264. ↓-42 (222) : Rosario+Vampire Season II 10 – Viz Shonen Jump Advanced, Nov 2012 [46.4] ::
362. ↑596 (958) : Bunny Drop 7 – Yen Press, Nov 2012 [32.8] ::
128. ↑29 (157) : Omamori Himari 9 – Yen Press, Nov 2012 [85.5] ::
2499. ↓-346 (2153) : Soul Eater 11 – Yen Press, Nov 2012 [0.0] ::
. : Soulless 2 – Yen Press, Nov 2012 [0.0] ::
1923. ↓-148 (1775) : Spice & Wolf (manga) 7 – Yen Press, Nov 2012 [0.1] ::
December Releases
. (last ranked 5 Aug 12) : Neon Genesis Evangelion The Shinji Ikari Raising Project 12 – Dark Horse, Dec 2012 [0.0] ::
. (last ranked 10 Jun 12) : Mizuki 1 – DMP, Dec 2012 [0.0] ::
. (last ranked 10 Jun 12) : Mizuki 2 – DMP, Dec 2012 [0.0] ::
1053. ↓-235 (818) : Caramel – DMP Juné, Dec 2012 [3.7] ::
1854. ↓-120 (1734) : I’ve Seen It All 2 – DMP Juné, Dec 2012 [0.1] ::
1261. ↑40 (1301) : Love Makes Everything Right – DMP Juné, Dec 2012 [1.6] ::
1365. ↓-296 (1069) : Ninth Life Love – DMP Juné, Dec 2012 [0.8] ::
918. ↑238 (1156) : Love On The Job 1 – DMP Project H, Dec 2012 [6.0] ::
. : Attack on Titan 3 – Kodansha Comics, Dec 2012 [0.0] ::
204. ↑76 (280) : Battle Angel Alita Last Order 16 – Kodansha Comics, Dec 2012 [60.6] ::
1184. ↑627 (1811) : A Certain Scientific Railgun 6 – Seven Seas, Dec 2012 [2.3] ::
1401. ↑517 (1918) : Mayo Chiki! 1 – Seven Seas, Dec 2012 [0.7] ::
306. ↓-5 (301) : Black Bird 15 – Viz Shojo Beat, Dec 2012 [40.5] ::
1655. ↑62 (1717) : The Earl & The Fairy 4 – Viz Shojo Beat, Dec 2012 [0.2] ::
. : Bakuman 17 – Viz Shonen Jump, Dec 2012 [0.0] ::
1859. ↓-34 (1825) : Bleach 52 – Viz Shonen Jump, Dec 2012 [0.1] ::
1511. ↑17 (1528) : Bleach 53 – Viz Shonen Jump, Dec 2012 [0.3] ::
. (last ranked 15 Jul 12) : Nura: Rise of the Yokai Clan 12 – Viz Shonen Jump, Dec 2012 [0.0] ::
699. ↑615 (1314) : Yu-Gi-Oh! Zexal 2 – Viz Shonen Jump, Dec 2012 [12.2] ::
. : Tenjo Tenge: Full Contact Edition 10 – Viz Signature, Dec 2012 [0.0] ::
2217. ↑ (last ranked 5 Aug 12) : Vagabond VizBig Edition 11 – Viz Signature, Dec 2012 [0.1] ::
1018. ↑433 (1451) : Alice in the Country of Hearts My Fanatic Rabbit 1 – Yen Press, Dec 2012 [4.2] ::
217. ↓-45 (172) : Haruhi Suzumiya Melancholy of Haruhi Suzumiya 14 – Yen Press, Dec 2012 [57.3] ::
1305. ↑211 (1516) : Puella Magi Madoka Magica 3 – Yen Press, Dec 2012 [1.3] ::
January Releases
740. ↓-54 (686) : Ze 7 – 801 Media, Jan 2013 [10.6] ::
1924. ↑153 (2077) : Oh My Goddess! 43 – Dark Horse, Jan 2013 [0.1] ::
957. ↓-111 (846) : Ai no Kusabi (novel) 8 – DMP Juné, Jan 2013 [5.2] ::
456. ↓-13 (443) : The Tyrant Falls in Love 8 – DMP Juné, Jan 2013 [24.5] ::
. : Genshiken Omnibus 3 – Kodansha Comics, Jan 2013 [0.0] ::
111. ↓-6 (105) : Negima! 37 – Kodansha Comics, Jan 2013 [94.4] ::
230. ↓-42 (188) : Phoenix Wright Miles Edgeworth 4 – Kodansha Comics, Jan 2013 [53.5] ::
13. ↔0 (13) : Sailor Moon 9 – Kodansha Comics, Jan 2013 [286.2] ::
736. ↑154 (890) : Girl Friends: Complete Collection 2 – Seven Seas, Jan 2013 [10.8] ::
. (last ranked 10 Jun 12) : Gunslinger Girl Omnibus 6 – Seven Seas, Jan 2013 [0.0] ::
. (last ranked 22 Jul 12) : I Don’t Like You At All, Big Brother! vols 3-4 collection – Seven Seas, Jan 2013 [0.0] ::
. (last ranked 6 May 12) : Punch Up! 3 – SuBLime, Jan 2013 [0.0] ::
1598. ↑165 (1763) : Loveless 10 – Viz, Jan 2013 [0.2] ::
. : X vols 13-15 collection – Viz, Jan 2013 [0.0] ::
1070. ↑128 (1198) : Dengeki Daisy 11 – Viz Shojo Beat, Jan 2013 [3.5] ::
. : Jiu Jiu 3 – Viz Shojo Beat, Jan 2013 [0.0] ::
. (last ranked 15 Jul 12) : Otomen 14 – Viz Shojo Beat, Jan 2013 [0.0] ::
2167. ↑377 (2544) : Bleach 54 – Viz Shonen Jump, Jan 2013 [0.1] ::
877. ↓-120 (757) : Naruto 60 – Viz Shonen Jump, Jan 2013 [6.8] ::
. : Case Closed 45 – Viz Shonen Sunday, Jan 2013 [0.0] ::
. : Jormungand 10 – Viz Signature, Jan 2013 [0.0] ::
1679. ↓-194 (1485) : A Bride’s Story 4 – Yen Press, Jan 2013 [0.1] ::
1491. ↓-206 (1285) : Black Butler 12 – Yen Press, Jan 2013 [0.3] ::
So what are the highlights? I think special mention should be made of the new Nausicaa Box (a two-volume hardcover set, isbn 9781421550640, rel. 6 November) and the Pokemon box sets: Red & Blue (isbn 9781421550060; should already be available) and Gold & Silver (isbn 9781421550077, rel. 4 September) which together collect the 14 volumes of the Pokemon “Adventures” series. Also, as if for my birthday, Viz is releasing an 18 volume box set of Ouran High School Host Club.
Additonally, there are a number of omnibuses from Dark Horse, Kodansha and Seven Seas (hit ctrl-F & search Omnibus) as well as additional 3-in-1 collections from Viz. For collectors, or for those of us who are coming late to these series (includes Angelic Layer, Cardcaptor Sakura from Clamp, plus Genshiken, Negima, and Gunslinger Girl; and various 2-in-1 print releases of other titles from Seven Seas). The omnibus format usually represents a cost savings, a ‘big-gulp’ opportunity – or a sub-$20 grand introduction to a new series.
I make no promises, but now that I’m finally caught up on the charts, I’ll try to give you a weekly or bi-weekly peek like this, as was previously promised. Welcome back.
