Share
VIDEOS 1 TO 50
Cluster Computing and MapReduce Lecture 3
Cluster Computing and MapReduce Lecture 3
Published: 2007/08/28
Channel: Google Developers
05 May 2014: GFS
05 May 2014: GFS
Published: 2014/05/06
Channel: Geoffrey Challen
GFS MSc 2015
GFS MSc 2015
Published: 2015/07/06
Channel: sumithmoratuwa
Google File System
Google File System
Published: 2015/10/23
Channel: Audiopedia
硅谷之路 9:深入浅出Google File System
硅谷之路 9:深入浅出Google File System
Published: 2017/05/16
Channel: BitTiger
Google File System
Google File System
Published: 2015/04/01
Channel: Roman Khabirov
The Hadoop File System (GFS, MAP REDUCE, BIG TABLE)
The Hadoop File System (GFS, MAP REDUCE, BIG TABLE)
Published: 2016/11/28
Channel: Cliff Green
Inside a Google data center
Inside a Google data center
Published: 2014/12/17
Channel: G Suite
Hadoop Distributed File System (HDFS) Introduction
Hadoop Distributed File System (HDFS) Introduction
Published: 2013/01/29
Channel: Hortonworks
File Systems as Fast As Possible
File Systems as Fast As Possible
Published: 2016/05/03
Channel: Techquickie
硅谷之路 9 深入浅出Google File System
硅谷之路 9 深入浅出Google File System
Published: 2016/04/13
Channel: BitTiger
Google File System (GFS o GooFS)
Google File System (GFS o GooFS)
Published: 2012/05/11
Channel: Manuel de la Torre
The World’s Best File Naming System
The World’s Best File Naming System
Published: 2014/12/11
Channel: How To Archive
Google File System
Google File System
Published: 2016/02/02
Channel: Kevin Rutz
Modeling and Verifying Google File System
Modeling and Verifying Google File System
Published: 2015/11/09
Channel: IEEE PROJECTS 2016-2017
Google File System - parte 1
Google File System - parte 1
Published: 2013/08/16
Channel: PinguimVideos
The rise of cloud computing systems
The rise of cloud computing systems
Published: 2015/12/10
Channel: Association for Computing Machinery (ACM)
El Google File System
El Google File System
Published: 2012/05/30
Channel: JoseKstro
Sync File System API
Sync File System API
Published: 2014/10/14
Channel: Chrome OS Developer
Chromebook File System now has access to DropBox and more!
Chromebook File System now has access to DropBox and more!
Published: 2015/06/10
Channel: Levi Sauerbrei
Optimal Flash Provisioning for Cloud Storage Workloads in Google
Optimal Flash Provisioning for Cloud Storage Workloads in Google's Colossus File System
Published: 2014/06/24
Channel: hkoie
Hadoop Tutorial: Intro to HDFS
Hadoop Tutorial: Intro to HDFS
Published: 2012/10/31
Channel: InfoQ
Setting Sharing Permissions For Google Files and Docs
Setting Sharing Permissions For Google Files and Docs
Published: 2015/07/29
Channel: Gabriel Massicotte
What is local file system & What is HDFS
What is local file system & What is HDFS
Published: 2014/07/12
Channel: Durga Software Solutions
Google Drive SDK Live: Google Drive vs Traditional File Systems
Google Drive SDK Live: Google Drive vs Traditional File Systems
Published: 2012/08/13
Channel: Google Developers
Google File System - parte 2
Google File System - parte 2
Published: 2013/08/16
Channel: PinguimVideos
The Linux File System - Explained
The Linux File System - Explained
Published: 2012/11/17
Channel: Nixie Pixel
google file system research paper
google file system research paper
Published: 2017/04/25
Channel: Федор Жарыхин
CHRP-PCA M01 C02 VD Google File System
CHRP-PCA M01 C02 VD Google File System
Published: 2017/06/08
Channel: InurturePro Sme
File System - Telugu
File System - Telugu
Published: 2014/11/15
Channel: Spoken-Tutorial IIT Bombay
Google Drive to Download any file system and easy...
Google Drive to Download any file system and easy...
Published: 2016/12/10
Channel: Free Service ই
27 File System Architecture - 1
27 File System Architecture - 1
Published: 2013/04/10
Channel: kashyap B
Google File System - parte 3
Google File System - parte 3
Published: 2013/08/16
Channel: PinguimVideos
Лекция 16: Распределенные файловые системы. Google Filesystem. Основные особенности
Лекция 16: Распределенные файловые системы. Google Filesystem. Основные особенности
Published: 2016/03/27
Channel: НОУ ИНТУИТ
What is DFS in Hindi - Distributed File System - File server - Part 6
What is DFS in Hindi - Distributed File System - File server - Part 6
Published: 2011/10/13
Channel: JagvinderThind
PHP Front To Back [Part 19] - File System Functions
PHP Front To Back [Part 19] - File System Functions
Published: 2017/04/01
Channel: Traversy Media
Wuala - a distributed file system
Wuala - a distributed file system
Published: 2007/11/02
Channel: GoogleTechTalks
Advanced Sharing Setting  in Google Drive for Files and Folders
Advanced Sharing Setting in Google Drive for Files and Folders
Published: 2017/03/26
Channel: SavioVazTutorials
Chrome Apps Office Hours: Synchronized File System
Chrome Apps Office Hours: Synchronized File System
Published: 2013/03/13
Channel: Google Developers
Google Glass Android processes and file system
Google Glass Android processes and file system
Published: 2013/05/03
Channel: Siamak Ashrafi
The Reiser4 Filesystem
The Reiser4 Filesystem
Published: 2012/08/22
Channel: GoogleTalksArchive
Google Android Training File System  7/18
Google Android Training File System 7/18
Published: 2010/12/22
Channel: supertecacademy
Hadoop Distributed Filesystem (HDFS) And Its Features
Hadoop Distributed Filesystem (HDFS) And Its Features
Published: 2016/12/25
Channel: Code Complete-The spirit of coding
Лекция 18: Распределенные файловые системы. Google Filesystem
Лекция 18: Распределенные файловые системы. Google Filesystem
Published: 2016/03/27
Channel: НОУ ИНТУИТ
Help for hacked sites: File system damage assessment
Help for hacked sites: File system damage assessment
Published: 2013/03/12
Channel: Google Webmasters
Seattle Conference on Scalability: Lustre File System
Seattle Conference on Scalability: Lustre File System
Published: 2012/08/22
Channel: GoogleTalksArchive
[Real6410] 06.Embedded Linux File System 製作JFFS2檔案系統
[Real6410] 06.Embedded Linux File System 製作JFFS2檔案系統
Published: 2013/09/13
Channel: 吳吉米
MapReduce  e Sistemas de Arquivo Distribuídos - Bancos de Dados & Aplicações 2015 - Aula 5a
MapReduce e Sistemas de Arquivo Distribuídos - Bancos de Dados & Aplicações 2015 - Aula 5a
Published: 2015/06/17
Channel: André Santanchè
Trabalhando com Filesystem e Cloud Storage no Laravel 5
Trabalhando com Filesystem e Cloud Storage no Laravel 5
Published: 2015/04/18
Channel: School of Net
Convert Coordinates system ETM to UTM and export autocad file to google earth
Convert Coordinates system ETM to UTM and export autocad file to google earth
Published: 2016/11/08
Channel: mahmoud shaaban
NEXT
GO TO RESULTS [51 .. 100]

WIKIPEDIA ARTICLE

From Wikipedia, the free encyclopedia
Jump to: navigation, search
Google File System
Operating system Linux kernel
Type Distributed file system
License Proprietary

Google File System (GFS or GoogleFS) is a proprietary distributed file system developed by Google to provide efficient, reliable access to data using large clusters of commodity hardware. A new version of Google File System code named Colossus was released in 2010.[1][2]

Design[edit]

Google File System is designed for system-to-system interaction, and not for user-to-system interaction. The chunk servers replicate the data automatically.

GFS is enhanced for Google's core data storage and usage needs (primarily the search engine), which can generate enormous amounts of data that must be retained; Google File System grew out of an earlier Google effort, "BigFiles", developed by Larry Page and Sergey Brin in the early days of Google, while it was still located in Stanford. Files are divided into fixed-size chunks of 64 megabytes, similar to clusters or sectors in regular file systems, which are only extremely rarely overwritten, or shrunk; files are usually appended to or read. It is also designed and optimized to run on Google's computing clusters, dense nodes which consist of cheap "commodity" computers, which means precautions must be taken against the high failure rate of individual nodes and the subsequent data loss. Other design decisions select for high data throughputs, even when it comes at the cost of latency.

A GFS cluster consists of multiple nodes. These nodes are divided into two types: one Master node and a large number of Chunkservers. Each file is divided into fixed-size chunks. Chunk servers store these chunks. Each chunk is assigned a unique 64-bit label by the master node at the time of creation, and logical mappings of files to constituent chunks are maintained. Each chunk is replicated several times throughout the network, with the minimum being three, but even more for files that have high end-in demand or need more redundancy.

The Master server does not usually store the actual chunks, but rather all the metadata associated with the chunks, such as the tables mapping the 64-bit labels to chunk locations and the files they make up, the locations of the copies of the chunks, what processes are reading or writing to a particular chunk, or taking a "snapshot" of the chunk pursuant to replicate it (usually at the instigation of the Master server, when, due to node failures, the number of copies of a chunk has fallen beneath the set number). All this metadata is kept current by the Master server periodically receiving updates from each chunk server ("Heart-beat messages").

Permissions for modifications are handled by a system of time-limited, expiring "leases", where the Master server grants permission to a process for a finite period of time during which no other process will be granted permission by the Master server to modify the chunk. The modifying chunkserver, which is always the primary chunk holder, then propagates the changes to the chunkservers with the backup copies. The changes are not saved until all chunkservers acknowledge, thus guaranteeing the completion and atomicity of the operation.

Programs access the chunks by first querying the Master server for the locations of the desired chunks; if the chunks are not being operated on (i.e. no outstanding leases exist), the Master replies with the locations, and the program then contacts and receives the data from the chunkserver directly (similar to Kazaa and its supernodes).

Unlike most other file systems, GFS is not implemented in the kernel of an operating system, but is instead provided as a userspace library.

Performance[edit]

Deciding from benchmarking results,[3] when used with relatively small number of servers (15), the file system achieves reading performance comparable to that of a single disk (80–100 MB/s), but has a reduced write performance (30 MB/s), and is relatively slow (5 MB/s) in appending data to existing files. (The authors present no results on random seek time.) As the master node is not directly involved in data reading (the data are passed from the chunk server directly to the reading client), the read rate increases significantly with the number of chunk servers, achieving 583 MB/s for 342 nodes. Aggregating a large number of servers also allows big capacity, while it is somewhat reduced by storing data in three independent locations (to provide redundancy).

See also[edit]

References[edit]

  1. ^ "Google's Colossus Makes Search Real-Time by Dumping MapReduce", High Scalability (World Wide Web log), 2010-09-11 .
  2. ^ "Colossus: Successor to the Google File System (GFS)". SysTutorials. 2012-11-29. Retrieved 2016-05-10. 
  3. ^ Ghemawat, Gobioff & Leung 2003.

Bibliography[edit]

External links[edit]

  • "GFS: Evolution on Fast-forward", Queue, ACM .
  • "Google File System Eval, Part I", Storage mojo .

Disclaimer

None of the audio/visual content is hosted on this site. All media is embedded from other sites such as GoogleVideo, Wikipedia, YouTube etc. Therefore, this site has no control over the copyright issues of the streaming media.

All issues concerning copyright violations should be aimed at the sites hosting the material. This site does not host any of the streaming media and the owner has not uploaded any of the material to the video hosting servers. Anyone can find the same content on Google Video or YouTube by themselves.

The owner of this site cannot know which documentaries are in public domain, which has been uploaded to e.g. YouTube by the owner and which has been uploaded without permission. The copyright owner must contact the source if he wants his material off the Internet completely.

Powered by YouTube
Wikipedia content is licensed under the GFDL and (CC) license