What are you using to load/stage your big data?
I’m curious to know which software (messaging bus, etc) people are using to gather data and process it for their big data, low latency needs. Are there any newcomers in the group to threaten 29West (informatica) or Tibco’s (ftl) ?? How about AMQP based solutions ?? Or zeromq ?? Do products like flume have a roll here ??
Let’s hear about your experiences and software selection in this area.
In an earlier post you pointed out that there’s this ‘low-latency’ component to this group.
How much latency can you tolerate? If you’re not looking at low-latency, then flume should be the default.
If you’re looking at streaming data for ‘real time’ analysis. (or near real-time) that would be an interesting question. (And even if Hadoop/HBase would be a good fit in the first place.)
NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!