Created by Li Yuntao / @dust_jead
04/17/2015
Spring XD 是一个统一的、分布式、可扩展的服务。它用于数据获取、实时分析、批量处理、数据导出。
Java JDK7 或更高版本
Spring XD 有两种运行模式,single-node和multi-node
点对点通道通常用于文档消息因为他们保证只有一个消费者将消费每条消息。然而,对于测试、监控与调试,需要使用某种途径可以获取所有穿过通道的消息。
Wire tap 模式通过在信道中插入一个收件人列表,当消息流过时,分发给主信道和辅助信道。
在 Spring XD 中, 流被定义为由事件驱动的数据,从一个Source到Sink,中间可以经过任意数量的Processor.
流处理在XD容器中进行,而流的定义和部署经由 XD Admin Server 完成。
模块由类型分类,可以由名称和分类轻易的区分模块的作用和功能。
在当前版本,Module包括source, sink, processor和job
A simple linear stream consists of a sequence of modules. Typically an Input Source, (optional) Processing Steps, and an Output Sink. As a simple example consider the collection of data from an HTTP Source writing to a File Sink. Using the DSL the stream description is:
Each module may take parameters. The parameters supported by a module are defined by the module implementation. As an example the http source module exposes port setting which allows the data ingestion port to be changed from the default value.
Streams are composed of modules which encapsulate a unit of work into a reusable component. A job in Spring XD must also be implemented as a module.
Streams are composed of modules which encapsulate a unit of work into a reusable component. A job in Spring XD must also be implemented as a module.
Labels, Single quotes, Double quotes, Escaping
启动runtime;启动shell;创建一个简单的流:ticktock; 查看Admin UI
启动流:
xd:> stream create --name ticktock --definition "time | log" --deploy
销毁流:
xd:>stream destroy --name ticktock
基于twitter stream API 的实时分析
- Spring XD Guide
- HOWTO: Spring XD for Real Time Analytics With Twitter Example Code