`
mxsfengg
  • 浏览: 206165 次
  • 性别: Icon_minigender_1
  • 来自: 广州
社区版块
存档分类
最新评论

（转）scribe java客户端 例子

Javalog4jApacheAnt.net 
阅读更多

上一篇文章已经安装完scribe，下面我们用java端，通过log4j 把日志写入scribe 日志系统。

 

 

一、生成scribe客户端

  • 生成 java客户端api

   A. 修改配置文件scribe.thrift
      cd /usr/local/scribeInstall/scribe/if
      vi scribe.thrift
      修改scribe.thrift文件: 把 include "fb303/if/fb303.thrift" 改成
      include "[thrift解压路径]/thrift-0.5.0/contrib/fb303/if/fb303.thrift"
   B. 生成 java客户端api     
      运行命令 thrift --gen java scribe.thrift
      运行后会生成一个‘gen-java’的文件夹.里面会有3个java类，封装了所有java客户端发送log所需要的api。

 

 

  • 生成客户端代码所需要的jar包

    A. 如果之前没设置ANT_HOME 和 PATH 请先设置这两个环境变量
      export ANT_HOME=/usr/local/apache-ant-1.8.0
      export PATH=$PATH:$ANT_HOME/bin
   B. 生成libthrift.jar
      cd /usr/local/scribeInstall/thrift-0.2.0/lib/java
      ant 
      (如果没有错误在本文夹夹下会生成libthrift.jar)
   C. 生成libfb303.jar

 

  cd /usr/local/scribeInstall/thrift-0.2.0/contrib/fb303/java

ant

ant 执行成功后 libfb303.jar 会出现在/usr/local/scribeInstall/contrib/fb303/java/build/lib下

 

 

二、创建项目，运行测试 

  • 创建项目
     A.在eclipse 创建普通java项目
     B.在项目中导入以下jar
     
    C.在项目中添加gen-java 文件夹里的三个java类。
  •    编写项目
    编写log4j 的scribe appender 

    AsyncScribeAppender.java:
    
Java代码
  1. package  com.logtest;  
  2.   
  3. import  org.apache.log4j.AsyncAppender;  
  4. /**  
  5.  * log4j 的scribe appender  
  6.  * 用ScribeAppender 类连接scribe服务器，并把日志写如scribe  
  7.  * @author ninja  
  8.  */   
  9. public   class  AsyncScribeAppender  extends  AsyncAppender {  
  10.   
  11.     private  String hostname;  
  12.     private  String scribeHost;  
  13.     private   int  scribePort;  
  14.     private  String scribeCategory;  
  15.     private  String encoading;  
  16.   
  17.     public  String getHostname() {  
  18.         return  hostname;  
  19.     }  
  20.   
  21.     public   void  setHostname(String hostname) {  
  22.         this .hostname = hostname;  
  23.     }  
  24.   
  25.     public  String getScribeHost() {  
  26.         return  scribeHost;  
  27.     }  
  28.   
  29.     public   void  setScribeHost(String scribeHost) {  
  30.         this .scribeHost = scribeHost;  
  31.     }  
  32.   
  33.     public   int  getScribePort() {  
  34.         return  scribePort;  
  35.     }  
  36.   
  37.     public   void  setScribePort( int  scribePort) {  
  38.         this .scribePort = scribePort;  
  39.     }  
  40.   
  41.     public  String getScribeCategory() {  
  42.         return  scribeCategory;  
  43.     }  
  44.   
  45.     public   void  setScribeCategory(String scribeCategory) {  
  46.         this .scribeCategory = scribeCategory;  
  47.     }  
  48.   
  49.     public  String getEncoading() {  
  50.         return  encoading;  
  51.     }  
  52.   
  53.     public   void  setEncoading(String encoading) {  
  54.         this .encoading = encoading;  
  55.     }  
  56.   
  57.     @Override   
  58.     public   void  activateOptions() {  
  59.         super .activateOptions();  
  60.         synchronized  ( this ) {  
  61.             ScribeAppender scribeAppender = new  ScribeAppender();  
  62.             scribeAppender.setLayout(getLayout());  
  63.             scribeAppender.setHostname(getHostname());  
  64.             scribeAppender.setScribeHost(getScribeHost());  
  65.             scribeAppender.setScribePort(getScribePort());  
  66.             scribeAppender.setScribeCategory(getScribeCategory());  
  67.             scribeAppender.setEncoding(getEncoading());  
  68.             scribeAppender.activateOptions();  
  69.             addAppender(scribeAppender);  
  70.         }  
  71.     }  
  72.   
  73.     @Override   
  74.     public   boolean  requiresLayout() {  
  75.         return   true ;  
  76.     }  
  77.   
  78. }   
package com.logtest;

import org.apache.log4j.AsyncAppender;
/**
 * log4j 的scribe appender
 * 用ScribeAppender 类连接scribe服务器，并把日志写如scribe
 * @author ninja
 */
public class AsyncScribeAppender extends AsyncAppender {

	private String hostname;
	private String scribeHost;
	private int scribePort;
	private String scribeCategory;
	private String encoading;

	public String getHostname() {
		return hostname;
	}

	public void setHostname(String hostname) {
		this.hostname = hostname;
	}

	public String getScribeHost() {
		return scribeHost;
	}

	public void setScribeHost(String scribeHost) {
		this.scribeHost = scribeHost;
	}

	public int getScribePort() {
		return scribePort;
	}

	public void setScribePort(int scribePort) {
		this.scribePort = scribePort;
	}

	public String getScribeCategory() {
		return scribeCategory;
	}

	public void setScribeCategory(String scribeCategory) {
		this.scribeCategory = scribeCategory;
	}

	public String getEncoading() {
		return encoading;
	}

	public void setEncoading(String encoading) {
		this.encoading = encoading;
	}

	@Override
	public void activateOptions() {
		super.activateOptions();
		synchronized (this) {
			ScribeAppender scribeAppender = new ScribeAppender();
			scribeAppender.setLayout(getLayout());
			scribeAppender.setHostname(getHostname());
			scribeAppender.setScribeHost(getScribeHost());
			scribeAppender.setScribePort(getScribePort());
			scribeAppender.setScribeCategory(getScribeCategory());
			scribeAppender.setEncoding(getEncoading());
			scribeAppender.activateOptions();
			addAppender(scribeAppender);
		}
	}

	@Override
	public boolean requiresLayout() {
		return true;
	}

}
 
 ScribeAppender .java

 

Java代码
  1. package  com.logtest;  
  2.   
  3. import  net.scribe.LogEntry;  
  4. import  net.scribe.scribe;  
  5.   
  6. import  org.apache.log4j.WriterAppender;  
  7. import  org.apache.log4j.spi.LoggingEvent;  
  8. import  org.apache.thrift.protocol.TBinaryProtocol;  
  9. import  org.apache.thrift.transport.TFramedTransport;  
  10. import  org.apache.thrift.transport.TSocket;  
  11. import  org.apache.thrift.transport.TTransportException;  
  12.   
  13. import  java.util.List;  
  14. import  java.util.ArrayList;  
  15. import  java.net.Socket;  
  16. import  java.net.UnknownHostException;  
  17. import  java.net.InetAddress;  
  18. import  java.io.IOException;  
  19.   
  20. /**  
  21.  * 继承WriterAppender   
  22.  * 实现了scribe 服务器的链接和日志的发送。  
  23.  * @author ninja  
  24.  */   
  25. public   class  ScribeAppender  extends  WriterAppender {  
  26.   
  27.     private  String hostname;  
  28.     private  String scribeHost;  
  29.     private   int  scribePort;  
  30.     private  String scribeCategory;  
  31.     private  String encoding;  
  32.   
  33.     private  List<LogEntry> logEntries;  
  34.   
  35.     private  scribe.Client client;  
  36.     private  TFramedTransport transport;  
  37.   
  38.     public  String getHostname() {  
  39.         return  hostname;  
  40.     }  
  41.   
  42.     public   void  setHostname(String hostname) {  
  43.         this .hostname = hostname;  
  44.     }  
  45.   
  46.     public  String getScribeHost() {  
  47.         return  scribeHost;  
  48.     }  
  49.   
  50.     public   void  setScribeHost(String scribeHost) {  
  51.         this .scribeHost = scribeHost;  
  52.     }  
  53.   
  54.     public   int  getScribePort() {  
  55.         return  scribePort;  
  56.     }  
  57.   
  58.     public   void  setScribePort( int  scribePort) {  
  59.         this .scribePort = scribePort;  
  60.     }  
  61.   
  62.     public  String getScribeCategory() {  
  63.         return  scribeCategory;  
  64.     }  
  65.   
  66.     public   void  setScribeCategory(String scribeCategory) {  
  67.         this .scribeCategory = scribeCategory;  
  68.     }  
  69.       
  70.   
  71.     public  String getEncoding() {  
  72.         return  encoding;  
  73.     }  
  74.   
  75.     public   void  setEncoding(String encoding) {  
  76.         this .encoding = encoding;  
  77.     }  
  78.   
  79.     /*  
  80.      * Activates this Appender by opening a transport to the Scribe server.  
  81.      */   
  82.     @Override   
  83.     public   void  activateOptions() {  
  84.         try  {  
  85.             synchronized  ( this ) {  
  86.                 if  (hostname ==  null ) {  
  87.                     try  {  
  88.                         hostname = InetAddress.getLocalHost()  
  89.                                 .getCanonicalHostName();  
  90.                     } catch  (UnknownHostException e) {  
  91.                         // can't get hostname   
  92.                     }  
  93.                 }  
  94.         System.out.println(scribeHost + scribePort + scribeCategory + encoding);  
  95.                 // Thrift boilerplate code   
  96.                 logEntries = new  ArrayList<LogEntry>( 1 );  
  97.                 TSocket sock = new  TSocket( new  Socket(scribeHost, scribePort));  
  98.                 transport = new  TFramedTransport(sock);  
  99.                 TBinaryProtocol protocol = new  TBinaryProtocol(transport,  
  100.                         false ,  false );  
  101.                 client = new  scribe.Client(protocol, protocol);  
  102.                 // This is commented out because it was throwing Exceptions for   
  103.                 // no good reason.   
  104.                 // transport.open();   
  105.             }  
  106.         } catch  (TTransportException e) {  
  107.             e.printStackTrace();  
  108.         } catch  (UnknownHostException e) {  
  109.             e.printStackTrace();  
  110.         } catch  (IOException e) {  
  111.             e.printStackTrace();  
  112.         } catch  (Exception e) {  
  113.             e.printStackTrace();  
  114.         }  
  115.     }  
  116.   
  117.     /*  
  118.      * Appends a log message to Scribe  
  119.      */   
  120.     @Override   
  121.     public   void  append(LoggingEvent event) {  
  122.         synchronized  ( this ) {  
  123.             try  {  
  124.                 String message = String.format("%s %s" , hostname, layout  
  125.                         .format(event));  
  126.                 LogEntry entry = new  LogEntry(scribeCategory, message);  
  127.                 logEntries.add(entry);  
  128.                 client.Log(logEntries);  
  129.             } catch  (Exception e) {  
  130.                 e.printStackTrace();  
  131.             } finally  {  
  132.                 logEntries.clear();  
  133.             }  
  134.         }  
  135.     }  
  136.   
  137.     @Override   
  138.     public   void  close() {  
  139.         if  (transport !=  null ) {  
  140.             transport.close();  
  141.         }  
  142.     }  
  143.   
  144.     @Override   
  145.     public   boolean  requiresLayout() {  
  146.         return   true ;  
  147.     }  
  148. }   
package com.logtest;

import net.scribe.LogEntry;
import net.scribe.scribe;

import org.apache.log4j.WriterAppender;
import org.apache.log4j.spi.LoggingEvent;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.transport.TFramedTransport;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransportException;

import java.util.List;
import java.util.ArrayList;
import java.net.Socket;
import java.net.UnknownHostException;
import java.net.InetAddress;
import java.io.IOException;

/**
 * 继承WriterAppender 
 * 实现了scribe 服务器的链接和日志的发送。
 * @author ninja
 */
public class ScribeAppender extends WriterAppender {

	private String hostname;
	private String scribeHost;
	private int scribePort;
	private String scribeCategory;
	private String encoding;

	private List<LogEntry> logEntries;

	private scribe.Client client;
	private TFramedTransport transport;

	public String getHostname() {
		return hostname;
	}

	public void setHostname(String hostname) {
		this.hostname = hostname;
	}

	public String getScribeHost() {
		return scribeHost;
	}

	public void setScribeHost(String scribeHost) {
		this.scribeHost = scribeHost;
	}

	public int getScribePort() {
		return scribePort;
	}

	public void setScribePort(int scribePort) {
		this.scribePort = scribePort;
	}

	public String getScribeCategory() {
		return scribeCategory;
	}

	public void setScribeCategory(String scribeCategory) {
		this.scribeCategory = scribeCategory;
	}
	

	public String getEncoding() {
		return encoding;
	}

	public void setEncoding(String encoding) {
		this.encoding = encoding;
	}

	/*
	 * Activates this Appender by opening a transport to the Scribe server.
	 */
	@Override
	public void activateOptions() {
		try {
			synchronized (this) {
				if (hostname == null) {
					try {
						hostname = InetAddress.getLocalHost()
								.getCanonicalHostName();
					} catch (UnknownHostException e) {
						// can't get hostname
					}
				}
		System.out.println(scribeHost + scribePort + scribeCategory + encoding);
				// Thrift boilerplate code
				logEntries = new ArrayList<LogEntry>(1);
				TSocket sock = new TSocket(new Socket(scribeHost, scribePort));
				transport = new TFramedTransport(sock);
				TBinaryProtocol protocol = new TBinaryProtocol(transport,
						false, false);
				client = new scribe.Client(protocol, protocol);
				// This is commented out because it was throwing Exceptions for
				// no good reason.
				// transport.open();
			}
		} catch (TTransportException e) {
			e.printStackTrace();
		} catch (UnknownHostException e) {
			e.printStackTrace();
		} catch (IOException e) {
			e.printStackTrace();
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

	/*
	 * Appends a log message to Scribe
	 */
	@Override
	public void append(LoggingEvent event) {
		synchronized (this) {
			try {
				String message = String.format("%s %s", hostname, layout
						.format(event));
				LogEntry entry = new LogEntry(scribeCategory, message);
				logEntries.add(entry);
				client.Log(logEntries);
			} catch (Exception e) {
				e.printStackTrace();
			} finally {
				logEntries.clear();
			}
		}
	}

	@Override
	public void close() {
		if (transport != null) {
			transport.close();
		}
	}

	@Override
	public boolean requiresLayout() {
		return true;
	}
}

 

 

   log4j.properties

 

Java代码
  1. # 1  \u5b9a\u4e49\u4e86\u4e24\u4e2a\u8f93\u51fa\u7aef  
  2. log4j.rootLogger = DEBUG,CONSOLE,scribe  
  3. log4j.addivity.org.apache=true   
  4.   
  5. log4j.appender.CONSOLE = org.apache.log4j.ConsoleAppender  
  6. log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout  
  7. log4j.appender.CONSOLE.layout.ConversionPattern = %-4r [%t] %-5p %c - %m%n  
  8.   
  9. log4j.logger.com.vmars= DEBUG, scribe  
  10. log4j.appender.scribe= com.logtest.AsyncScribeAppender  
  11. log4j.appender.scribe.encoading=utf-8   
  12. log4j.appender.scribe.hostname=scribe  
  13. log4j.appender.scribe.scribeHost=192.168 . 2.221   
  14. log4j.appender.scribe.scribePort=1463   
  15. log4j.appender.scribe.hostname=ninja  
  16. log4j.appender.scribe.scribeCategory=scribe  
  17. log4j.appender.scribe.layout=org.apache.log4j.PatternLayout  
  18. log4j.appender.scribe.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2 }: %m%n   
#1 \u5b9a\u4e49\u4e86\u4e24\u4e2a\u8f93\u51fa\u7aef
log4j.rootLogger = DEBUG,CONSOLE,scribe
log4j.addivity.org.apache=true

log4j.appender.CONSOLE = org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern = %-4r [%t] %-5p %c - %m%n

log4j.logger.com.vmars= DEBUG, scribe
log4j.appender.scribe= com.logtest.AsyncScribeAppender
log4j.appender.scribe.encoading=utf-8
log4j.appender.scribe.hostname=scribe
log4j.appender.scribe.scribeHost=192.168.2.221
log4j.appender.scribe.scribePort=1463
log4j.appender.scribe.hostname=ninja
log4j.appender.scribe.scribeCategory=scribe
log4j.appender.scribe.layout=org.apache.log4j.PatternLayout
log4j.appender.scribe.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n

 

 

   测试客户端：

 

Java代码
  1. package  com.logtest;  
  2.   
  3. import  org.apache.commons.logging.Log;  
  4. import  org.apache.commons.logging.LogFactory;  
  5.   
  6.   
  7. public   class  LogTest {  
  8.       
  9.     private   static  Log log = LogFactory.getLog(LogTest. class );  
  10.       
  11.     public   static   void  main(String[] args) {  
  12.         log.error("this is a charactor test " );  
  13.         log.debug("这是中文测试" );  
  14.         log.fatal("fatal error 致命错误！！" );  
  15.     }  
  16. } 
分享到：
| （转）scribe 安装过程
评论
3 楼 雨中淡月 2012-11-23  
如果我想自己写一个类似于example1.conf，为什么运行的时候报错呢?
2 楼 sankby 2011-10-08  
没有人在用scribe吗？
1 楼 sankby 2011-09-29  
我在suse上装了scribe，使用example1.conf启动，
echo  "hello world"|scribe_home/examples/scribe_cat test  这个命令测试,能在/tmp/scribetest/test/test_current  看到日志内容。
如果用你这个测试类运行之后没有报错，但是日志写到哪里去了？test_current  没有看到.
发表评论

您还没有登录,请您登录后再发表评论

相关推荐

    scribe客户端所需jar包

    scribe客户端所需要用到的jar包，里面还包含一个java客户端用log4j向scribe服务端发送日志的小例子

    Oauth实例(使用Scribe-java)

    Oauth实例(使用Scribe-java实现新浪微博的Oauth)

    fb303，使用Java连接Scribe必备的JAR包

    包含编译好的FacebookService, fb_status, FacebookBase, LogEntry, ResultCode, scribe类。 按照完thrift和scribe后，就不用自己生成了。

    scribe软件

    scribe软件

    scribe-log4j:将日志消息发送到 Scribe 的 log4j appender

    scribe-client - 为 scribe 自动生成的 Java 客户端 scribe-log4j - scribe-log4j附加程序 scribe-example - 使用示例 入门 克隆仓库 git clone https://github.com/caiiiycuk/scribe-log4j.git 将 scribe-client ...

    scribe-1.3.5.jar

    scribe-1.3.5.jar

    scribe详细安装文档

    很好的东西，描述如何详细的安装scribe，编写安装脚本

    facebook scribe

    facebook的log系统 - scribe

    VideoScribe.7z

    VideoScribe.exeVideoScribe.exeVideoScribe.exeVideoScribe.exeVideoScribe.exeVideoScribe.exe

    VideoScribe.3.2.003

    如果您想自己尝试一下，则可以以某种方式安装摄像机并花费数小时进行绘制，但是更有效的方法是使用VideoScribe。它使您可以创建自己绘制的视频演示文稿。界面的现代设计及其直观的控件使此过程尽可能简单。 该软件...

    scribe-2.2.tar.gz

    scribe-2.2.tar.gz 下载地址： https://github.com/facebook/scribe Scribe is a server for aggregating log data streamed in real time from a large number of servers. It is designed to be scalable, ...

    Flat earth scribe

    Flat earth scribe

    VideoScribe

    VideoScribe手绘视频制作软件

    Facebook大量数据处理 Scribe.zip

    Facebook大量数据处理 Scribe ,Scribe旨在帮助Facebook处理服务器上的大量数据，正像Scribe网页所述“如经常访问Fa...

    Scribe日志记录讲解

    thrift框架的分布式日志记录框架，支持多种语言实现。

    VideoScribe for mac版

    一款非常有趣的手绘动画视频制作软件，我们经常会看到一些创意动画视频，其中就有一只拿着画笔的手画出整幅画面，就像快进效果的沙画一样。这种手绘视频不仅有创意，还可以装X，自己可以制定视频内容，可以是你跟你...

    scribe-2.1.tar.gz

    Scribe is a server for aggregating log data streamed in real time from a large number of servers. It is designed to be scalable, extensible without client-side modification, and robust to failure of ...

    scribe-version-2.01.tar.gz

    Scribe is a server for aggregating log data streamed in real time from a large number of servers. It is designed to be scalable, extensible without client-side modification, and robust to failure of ...

    scribe-apache-1.5.0.zip

    junrar.zip,普通javaplainjavaunrar util中的rar解压库（前sf项目）

    scribe+hadoop+log4j+hive+mysql

    一个月研究时间试验后总结的文档用户系统日志数据分析 比较实用

Magicbox
Global site tag (gtag.js) - Google Analytics