展会信息港展会大全

Giraph基础架构介绍(图解)
来源:互联网   发布日期:2016-01-26 10:44:14   浏览:1879次  

导读:Giraph介绍: Apache Giraph is an iterative graph processing system built for high scalability For example, it is currently used at Facebook to analyze the social graph ...

Giraph介绍:

Apache Giraph is an iterative graph processing system built for high scalability. For example, it is currently used at Facebook to analyze the social graph formed by users and their connections. Giraph originated as the open-source counterpart to Pregel, the graph processing architecture developed at Google and described in a 2010 paper. Both systems are inspired by the Bulk Synchronous Parallel model of distributed computation introduced by Leslie Valiant. Giraph adds several features beyond the basic Pregel model, including master computation, sharded aggregators, edge-oriented input, out-of-core computation, and more. With a steady development cycle and a growing community of users worldwide, Giraph is a natural choice for unleashing the potential of structured datasets at a massive scale.

原理:Giraph基于Hadoop而建,将MapReduce中Mapper进行封装,未使用reducer。在Mapper中进行多次迭 代,每次迭代等价于BSP模型中的SuperStep。一个Hadoop Job等价于一次BSP作业。基础结构如下图所示。

Giraph 的基础架构

每部分的功能如下:

(1)ZooKeeper: responsible for computation state

partition/worker mapping

global state: #superstep

checkpoint paths, aggregator values, statistics

(2) Master: responsible for coordination

assigns partitions to workers

coordinates synchronization

requests checkpoints

aggregates aggregator values

collects health statuses

(3) Worker: responsible for vertices

invokes active vertices compute() function

sends, receives and assigns messages

computes local aggregation values

源码。

赞助本站

人工智能实验室

相关热词: Giraph

AiLab云推荐
展开

热门栏目HotCates

Copyright © 2010-2024 AiLab Team. 人工智能实验室 版权所有    关于我们 | 联系我们 | 广告服务 | 公司动态 | 免责声明 | 隐私条款 | 工作机会 | 展会港