Big data integration firm Splunk has updated its software to make it faster and increase its reliability.
The firm, which specialises in collecting and monitoring large amounts of machine generated data, believes that reliability will cut users’ costs, by obviating the need to keep multiple copies of huge datasets, and encourage more people to give Splunk a bigger role in their Big Data projects.
“If you can turn it into data, you can put it into Splunk,” said D J Skillman, Splunk’s European technical director. “We are very flexible on the inject side, and solve the massive problem of data collection.”
Splunk connects to other Big Data tools, but wants to expand its role beyond collection and, where possible, become the basic system used to hold and manipulate the data. Splunk 5 includes new “platform” style features to encourage developers to do more within the tool, such as software development kits (SDKs) for Python, JavaScript and PHP.
“Resilience used to be built in by making copies and using a SAN [storage area network], but if you are collecting 5Tbyte a day in a SAN environment, the cost is astronomical,” he said. Building in resiliency in the software means the same performance and reliability can be done on commodity hardware, he said.
Splunk is not trying to take on relational databases – Skillman says they are still best for old-style transactional data like customer records, and it also leaves a lot of the “unstructured” data such as voice and video to other players, but concentrates on users who generate masses of text data from machine sources.
Splunk also now has a bi-directional connector to Hadoop, which Skillman says is, again, about allowing for use of the best tool for the job. “Hadoop is very inexpensive, and massively flexible for large dataset batch processing, but you have to program it.” Users need to have big data scientists to get the most out of Hadoop, he said.
Splunk is not open source, but operates on a “freemium” model whereby users can download and use the software for free, until the amount of data stored each day gets larger than 400MB. Above that, detailed pricing isn’t published, but is understood to start at $10,000 for a licence to store 1GB a day, falling rapidly as volumes rise.
Do you know about Europe’s role in tech history? Try our quiz!
Tesla lays off software, service, engineering staff after disbanding Supercharger team, as major cull continues
Dominant Bitcoin ETF Grayscale Bitcoin Trust shows first net inflow since January as investors flock…
US campaign funding groups backed by cryptocurrency sector raise more than $102m as firms seek…
Explore the cutting-edge realm of cybersecurity with 'A New Age of Cybersecurity' podcast. Learn how…
Robinhood Markets says it received SEC enforcement notice over cryptocurrency trading platform amidst ongoing crackdown
Chip designer Synopsys to sell software integrity unit to private investors to create new independent…