NVIDIA Tesla V100 GPU Accelerators Are Great Gear For Great AI Researchers. That’s Why Our CEO Jensen

How To Buy Bitcoins In India | A Step-By-Step Guide Find more Bitcoin mining rig reviews: http://bitcoinist.net

Introducing SAMOA, an open source platform for mining big data streams.

https://github.com/yahoo/samoa

Machine learning and data mining are well established techniques in the world of IT and especially among web companies and startups. Spam detection, personalization and recommendations are just a few of the applications made possible by mining the huge quantity of data available nowadays. However, “big data” is not only about Volume, but also about Velocity (and Variety, 3V of big data).

The usual pipeline for modeling data (what “data scientists” do) involves taking a sample from production data, cleaning and preprocessing it to make it usable, training a model for the task at hand and finally deploying it to production. The final output of this process is a pipeline that needs to run periodically (and be maintained) in order to keep the model up to date. Hadoop and its ecosystem (e.g., Mahout) have proven to be an extremely successful platform to support this process at web scale.

However, no solution is perfect and big data is “data whose characteristics forces us to look beyond the traditional methods that are prevalent at the time”. The current challenge is to move towards analyzing data as soon as it arrives into the system, nearly in real-time.

For example, models for mail spam detection get outdated with time and need to be retrained with new data. New data (i.e., spam reports) comes in continuously and the model starts being outdated the moment it is deployed: all the new data is sitting without creating any value until the next model update. On the contrary, incorporating new data as soon as it arrives is what the “Velocity” in big data is about. In this case, Hadoop is not the ideal tool to cope with streams of fast changing data.

Distributed stream processing engines are emerging as the platform of choice to handle this use case. Examples of these platforms are Storm, S4, and recently Samza. These platforms join the scalability of distributed processing with the fast response of stream processing. Yahoo has already adopted Storm as a key technology for low-latency big data processing.

Alas, currently there is no common solution for mining big data streams, that is, for doing machine learning on streams on a distributed environment.

Enter SAMOA

SAMOA (Scalable Advanced Massive Online Analysis) is a framework for mining big data streams. As most of the big data ecosystem, it is written in Java. It features a pluggable architecture that allows it to run on several distributed stream processing engines such as Storm and S4. SAMOA includes distributed algorithms for the most common machine learning tasks such as classification and clustering. For a simple analogy, you can think of SAMOA as Mahout for streaming.

SAMOA is both a platform and a library. As a platform, it allows the algorithm developer to abstract from the underlying execution engine, and therefore reuse their code to run on different engines. It also allows to easily write plug-in modules to port SAMOA to different execution engines.

As a library, SAMOA contains state-of-the-art implementations of algorithms for distributed machine learning on streams. The first alpha release allows classification and clustering.

For classification, we implemented a Vertical Hoeffding Tree (VHT), a distributed streaming version of decision trees tailored for sparse data (e.g., text). For clustering, we included a distributed algorithm based on CluStream. The library also includes meta-algorithms such as bagging.

HOW DOES IT WORK?

An algorithm in SAMOA is represented by a series of nodes communicating via messages along streams that connect pairs of nodes (a graph). Borrowing the terminology from Storm, this is called a Topology. Each node in the Topology is a Processor that sends messages to a Stream. The user code that implements the algorithm resides inside a Processor. Figure 3 shows an example of a Processor joining two stream from two source Processors. Here is a code snippet to build such a topology in SAMOA.

TopologyBuilder builder; Processor sourceOne = new SourceProcessor(); builder.addProcessor(sourceOne); Stream streamOne = builder.createStream(sourceOne); Processor sourceTwo = new SourceProcessor(); builder.addProcessor(sourceTwo); Stream streamTwo = builder.createStream(sourceTwo); Processor join = new JoinProcessor(); builder.addProcessor(join).connectInputShuffle(streamOne).connectInputKey(streamTwo);

SWEET! HOW DO I GET STARTED?

1. Download SAMOA

git clone git@github.com:yahoo/samoa.git cd samoa mvn -Pstorm package

2. Download the Forest CoverType dataset.

wget "http://downloads.sourceforge.net/project/moa-datastream/Datasets/Classification/covtypeNorm.arff.zip" unzip covtypeNorm.arff.zip

Forest CoverType contains the forest cover type for 30 x 30 meter cells obtained from US Forest Service (USFS) Region 2 Resource Information System (RIS) data. It contains 581,012 instances and 54 attributes, and it has been used in several papers on data stream classification.

3. Download a simple logging library.

wget "http://repo1.maven.org/maven2/org/slf4j/slf4j-simple/1.7.2/slf4j-simple-1.7.2.jar"

4. Run an Example. Classifying the CoverType dataset with the VerticalHoeffdingTree in local mode.

java -cp slf4j-simple-1.7.2.jar:target/SAMOA-Storm-0.0.1.jar com.yahoo.labs.samoa.DoTask "PrequentialEvaluation -l classifiers.trees.VerticalHoeffdingTree -s (ArffFileStream -f covtypeNorm.arff) -f 100000"

The output will be a sequence of the evaluation metrics for accuracy, taken every 100,000 instances.

To run the example on Storm, please refer to the instructions on the wiki.

I WANT TO KNOW MORE!

For more information about SAMOA, see the README and the wiki on github, or post a question on the mailing list.

SAMOA is licensed under an Apache Software License v2.0. You are welcome to contribute to the project! SAMOA accepts contributions under an Apache style contributor license agreement.

Good luck! We hope you find SAMOA useful. We will continue developing the framework by adding new algorithms and platforms.

Gianmarco De Francisci Morales (gdfm@yahoo-inc.com) and Albert Bifet (abifet@yahoo.com) @ Yahoo Labs Barcelona

Cyberpunk Street by Yoshimitszu

Tele-Present Water by David Bowen

I rarely use the phrase ‘mind blown’, but this is one of those rare occurrences.

An art installation which combines real-time data, mechanical puppetry, and a physical grid representation usually employed virtually with computers:

This installation draws information from the intensity and movement of the water in a remote location. Wave data is being collected in real-time from National Oceanic and Atmospheric Administration data buoy station 46246, 49.985 N 145.089 W (49°59'7" N 145°5'20" W) on the Pacific Ocean. The wave intensity and frequency is scaled and transferred to the mechanical grid structure resulting in a simulation of the physical effects caused by the movement of water from halfway around the world.

Link to the artist’s website for this work can be found here

Scrying Pen

Webtoy by Andy Matuschak uses neural network-trained SketchRNN dataset to visualize in realtime potential sketch marks whilst you are drawing particular objects:

This pen’s ink stretches backwards into the past and forwards into possible futures. The two sides make a strange loop: the future ink influences how you draw, which in turn becomes the new “past” ink influencing further future ink.

Put another way: this is a realtime implementation of SketchRNN which predicts future strokes while you draw.

Currently works best in Chrome, you can try it out for yourself here

Shape Representation by Zippables

Computational Fabrication research from the Interactive Geometry Lab can turn 3D model files into objects with textiles, connecting parts and forming shape using zip fasteners:

Fabrication from developable parts is the basis for arts such as papercraft and needlework, as well as modern architecture and CAD in general, and it has inspired much research. We observe that the assembly of complex 3D shapes created by existing methods often requires first fabricating many small parts and then carefully following instructions to assemble them together. Despite its significance, this error prone and tedious process is generally neglected in the discussion. We present the concept of zippables – single, two dimensional, branching, ribbon-like pieces of fabric that can be quickly zipped up without any instructions to form 3D objects. Our inspiration comes from the so-called zipit bags (just-zipit.com), which are made of a single, long ribbon with a zipper around its boundary. In order to assemble the bag, one simply needs to zip up the ribbon. Our method operates in the same fashion, but it can be used to approximate a wide variety of shapes. Given a 3D model, our algorithm produces plans for a single 2D shape that can be laser cut in few parts from fabric or paper. A zipper can then be attached along the boundary by sewing, or by gluing using a custom-built fastening rig. We show physical and virtual results that demonstrate the capabilities of our method and the ease with which shapes can be assembled. 

More Here

Quick, Draw! by Google

Quick, Draw! is an artificial intelligence experiment that looks for familiarizing yourself with how neural networks work to identify objects and text in photos 

Launching the Future of Space Communications

Our newest communications satellite, named the Tracking and Data Relay Satellite-M or TDRS-M, launches Aug. 18 aboard an Atlas V rocket from our Kennedy Space Center in Florida. It will be the 13th TDRS satellite and will replenish the fleet of satellites supporting the Space Network, which provides nearly continuous global communications services to more than 40 of our missions.

Communicating from space wasn’t always so easy. During our third attempt to land on the moon in 1970, the Apollo 13 crew had to abort their mission when the spacecraft’s oxygen tank suddenly exploded and destroyed much of the essential equipment onboard. Made famous in the movie ‘Apollo 13’ by Ron Howard and starring Tom Hanks, our NASA engineers on the ground talked to the crew and fixed the issue. Back in 1970 our ground crew could only communicate with their ground teams for 15 percent of their orbit – adding yet another challenge to the crew. Thankfully, our Apollo 13 astronauts survived and safely returned to Earth. 

Now, our astronauts don’t have to worry about being disconnected from their teams! With the creation of the TDRS program in 1973, space communications coverage increased rapidly from 15 percent coverage to 85 percent coverage. And as we’ve continued to add TDRS spacecraft, coverage zoomed to over 98 percent!

TDRS is a fleet of satellites that beam data from low-Earth-orbiting space missions to scientists on the ground. These data range from cool galaxy images from the Hubble Space Telescope to high-def videos from astronauts on the International Space Station! TDRS is operated by our Space Network, and it is thanks to these hardworking engineers and scientists that we can continuously advance our knowledge about the universe!  

What’s up next in space comm? Only the coolest stuff ever! LASER BEAMS. Our scientists are creating ways to communicate space data from missions through lasers, which have the ability to transfer more data per minute than typical radio-frequency systems. Both radio-frequency and laser comm systems send data at the speed of light, but with laser comm’s ability to send more data at a time through infrared waves, we can receive more information and further our knowledge of space.

How are we initiating laser comm? Our Laser Communications Relay Demonstration is launching in 2019! We’re only two short years away from beaming space data through lasers! This laser communications demo is the next step to strengthen this technology, which uses less power and takes up less space on a spacecraft, leaving more power and room for science instruments.

Watch the TDRS launch live online at 8:03 a.m. EDT on Aug. 18: https://www.nasa.gov/nasalive

Join the conversation on Twitter: @NASA_TDRS and @NASALasercomm!

Make sure to follow us on Tumblr for your regular dose of space: http://nasa.tumblr.com

Augmented reality climbing wall - Full video

YouTube Artifacts

Latest AR exhibition from MoMAR (who ran a guerilla show earlier this year) returns to the Pollock Room at MoMA New York featuring works by David Kraftsow, responsible for the YouTube Artififacts bot that regularly generates animated images from distorted videos:

Welcome to The Age of the Algorithm. A world in which automated processes are no longer simply tools at our disposal, but the single greatest omnipresent force currently shaping our world. For the most part, they remain unseen. Going about their business, mimicking human behavior and making decisions based on statistical analysis of what they ‘think’ is right. If the role of art in society is to incite reflection and ask questions about the state of our world, can algorithms be a part of determining and defining people’s artistic and cultural values? MoMAR presents a series of eight pieces created by David Kraftsow’s YouTube Artifact Bot.

More Here