tag:blogger.com,1999:blog-4446292666398344382.post1612501654655681281..comments2024-01-24T07:14:43.438-08:00Comments on Machined Learnings: Do you really have big data?Paul Mineirohttp://www.blogger.com/profile/05439062526157173163noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-4446292666398344382.post-14399682347256464052012-12-22T03:21:56.700-08:002012-12-22T03:21:56.700-08:00Thanks Paul.
The term 'Big Data' is a mis...Thanks Paul.<br /><br />The term 'Big Data' is a misnomer because there can be significant value in small data but as you say, it depends on the problem type.<br /><br />Wrt c), it would be useful if the industry began to classify what types of problems DL is superior at otherwise the religious war will continue but also confuse the heck out of organizations who want to do the 'right' thing.<br /><br />As an aside, it was good to see the analysis based on a computational constraint (of 27 secs) because an area that is rarely covered is the time to re-train (as new data becomes available) which must become ever more important as near real-time becomes a reality.Unknownhttps://www.blogger.com/profile/16570083132483784714noreply@blogger.comtag:blogger.com,1999:blog-4446292666398344382.post-82994244567557285292012-12-21T21:39:14.591-08:002012-12-21T21:39:14.591-08:00Mahout exemplifies the challenges that I/O can cre...Mahout exemplifies the challenges that I/O can create for distributed iterative computations.<br /><br />In general everyone agrees starting with a sample of data on a laptop for quick initial experimentation and visualization is sound procedure. Whether or not that's sufficient is context dependent. By analogy, many programs are written in scripting languages, and a few are written in assembly for maximum performance. Ad serving is so lucrative that minor improvements are economically valuable, but I suspect for many problems the predictor that arises from the subsampled data might be the final product.<br /><br />I'm not aware of any published comparison of the kind you seek.Paul Mineirohttps://www.blogger.com/profile/05439062526157173163noreply@blogger.comtag:blogger.com,1999:blog-4446292666398344382.post-59234196749871053982012-12-21T19:53:11.837-08:002012-12-21T19:53:11.837-08:00The overall message is basically "there's...The overall message is basically "there's no data like more data" is sometimes misleading because in practice there is a computational constraint which affects the types of learning methods that are feasible. Sometimes throwing away data to allow for more complicated learning methods is worth it.<br /><br />Specifically:<br /><br />a) I would say try deep learning (specifically neural networks) if you have a moderate sized data set (moderate = "fits on a single box and tolerably slow to train with multicores/gpus") and little intuition about what kinds of transformations would make your problem look more linear (aka, no intuition about a good kernel). You should also look at GBDTs since they work great in this zone.<br /><br />b) Yes constant factors do matter! But the important message is that network I/O is a major bottleneck in the distributed context. If NUMA becomes popular then the machines will start looking even more like little clusters so we can't just ignore this problem.<br /><br />c) Deep learning methods are superior to all other existing choices for particular types of problems. The key to success in a practical context is to understand the type of problem you face and which techniques are likely to work well.Paul Mineirohttps://www.blogger.com/profile/05439062526157173163noreply@blogger.comtag:blogger.com,1999:blog-4446292666398344382.post-14847358069685400922012-12-21T19:44:13.749-08:002012-12-21T19:44:13.749-08:00(Posted on behalf on Dinesh B Vadhia)
Hi Paul
T...(Posted on behalf on Dinesh B Vadhia)<br /><br />Hi Paul<br /> <br />There isn't an option to leave a comment without owning an identity through a 3rd party. Anyway, another terrific post.<br /> <br />Are the conclusions: <br />a) If you want predictions in a commercially (competitive) reasonable time then stick to traditional learning methods, if not then try deep learning?<br /> <br />b) Faster hardware helps both types of methods. <br /> <br />c) The intrinsic performance of deep learning algorithms have to improve by a significant factor to be commercially competitive with existing methods. <br /> <br />Is that about right?<br /> <br />Best ...<br />Dinesh<br />Paul Mineirohttps://www.blogger.com/profile/05439062526157173163noreply@blogger.comtag:blogger.com,1999:blog-4446292666398344382.post-45635897144949087502012-12-21T02:42:17.678-08:002012-12-21T02:42:17.678-08:00I wasn't aware there was a war brewing but I h...I wasn't aware there was a war brewing but I have been wondering about the value of Mahout in certain scenario's. I've seen people use Mahout on clusters where I thought other tools (VW) on my laptop could have done the trick.<br /><br />Has anyone done a proper comparison of certain use cases that you know of?Jurgen Van Gaelhttps://www.blogger.com/profile/16153896873199339437noreply@blogger.comtag:blogger.com,1999:blog-4446292666398344382.post-74298964189340272482012-12-21T00:19:54.839-08:002012-12-21T00:19:54.839-08:00Yup. In contrast, a lot of natural language proce...Yup. In contrast, a lot of natural language processing really seems to benefit from more data. It just really depends.<br /><br />However, even if the actual model estimation is done with a relatively small dataset, you still might have to use a cluster to quickly apply the model to the rest of the dataset. That's the situation I ran into.jfolsonhttps://www.blogger.com/profile/17741840871174299587noreply@blogger.com