Facebook and memcached - Tech Talk

Channel: FacebookDevelopers   |   2013/11/27
Play Video
1
Facebook and memcached - Tech Talk
Facebook and memcached - Tech Talk
::2013/11/27::
Play Video
2
Memcache Basics
Memcache Basics
::2013/02/05::
Play Video
3
Scaling your Website with Memcached, discussion with Northscale cofounder
Scaling your Website with Memcached, discussion with Northscale cofounder
::2010/04/07::
Play Video
4
15. Memcached
15. Memcached
::2012/08/10::
Play Video
5
Improving Website Performance and Scalability with Memcached [October 17, 2013]
Improving Website Performance and Scalability with Memcached [October 17, 2013]
::2013/10/18::
Play Video
6
9  ROR ScalingRails   Memcached fixed
9 ROR ScalingRails Memcached fixed
::2013/12/13::
Play Video
7
Secure Cloud Server Setup: Installing MemCache [Part 9]
Secure Cloud Server Setup: Installing MemCache [Part 9]
::2013/09/09::
Play Video
8
7 Database management   MemCacheD 008 PHP MySQL Integration
7 Database management MemCacheD 008 PHP MySQL Integration
::2013/09/03::
Play Video
9
Using Memcached with Python
Using Memcached with Python
::2010/08/18::
Play Video
10
Scaling Facebook with OpenSource tools
Scaling Facebook with OpenSource tools
::2010/02/14::
Play Video
11
Agile Memcached
Agile Memcached
::2012/10/26::
Play Video
12
Big Data: Hadoop and Memcached
Big Data: Hadoop and Memcached
::2014/04/04::
Play Video
13
Using Couchbase Server to Solve Memcached Challenges
Using Couchbase Server to Solve Memcached Challenges
::2013/01/07::
Play Video
14
6 Database management   MemCacheD PHP File Includes
6 Database management MemCacheD PHP File Includes
::2013/09/03::
Play Video
15
Memcached Functions in MySQL
Memcached Functions in MySQL
::2009/11/28::
Play Video
16
1 Database management   MemCacheD   MemCacheD Features
1 Database management MemCacheD MemCacheD Features
::2013/09/03::
Play Video
17
WordPress with nginx + memcached -- without PHP
WordPress with nginx + memcached -- without PHP
::2011/09/13::
Play Video
18
Data at Scale A Tale of MongoDB, Riak, Redis, Memcached - Mark Uhrmacher, Founding CTO of Ideeli
Data at Scale A Tale of MongoDB, Riak, Redis, Memcached - Mark Uhrmacher, Founding CTO of Ideeli
::2013/11/06::
Play Video
19
Garantia Data Goes GA with Cloud-based Redis and Memcached Platform
Garantia Data Goes GA with Cloud-based Redis and Memcached Platform
::2013/02/14::
Play Video
20
Gear6 makes memcached infrastructure lower cost on cloud
Gear6 makes memcached infrastructure lower cost on cloud
::2010/04/14::
Play Video
21
Fragment Caching and Memcached
Fragment Caching and Memcached
::2011/03/27::
Play Video
22
Πώς να εγκαταστήσετε το memcached (XAMPP)
Πώς να εγκαταστήσετε το memcached (XAMPP)
::2012/12/11::
Play Video
23
OutputCache com Memcached
OutputCache com Memcached
::2012/05/19::
Play Video
24
Cluster Apache + PHP + Memcached - Taller III STR Sistemas
Cluster Apache + PHP + Memcached - Taller III STR Sistemas
::2013/04/23::
Play Video
25
8   3   Example Memcached 1637)
8 3 Example Memcached 1637)
::2013/07/28::
Play Video
26
Big Data: Hadoop and Memcached
Big Data: Hadoop and Memcached
::2014/02/07::
Play Video
27
How to Run Memcached as a Windows Service
How to Run Memcached as a Windows Service
::2013/10/13::
Play Video
28
Memcached para aplicação escalável
Memcached para aplicação escalável
::2013/09/19::
Play Video
29
3 Database management   MemCacheD Cache Synchronization
3 Database management MemCacheD Cache Synchronization
::2013/09/03::
Play Video
30
Hosting PHP 5.4 OPcache Memcached
Hosting PHP 5.4 OPcache Memcached
::2014/03/05::
Play Video
31
Brian Stanley: Memcached: cachea TODO
Brian Stanley: Memcached: cachea TODO
::2014/01/23::
Play Video
32
Instalando Memcached segunda parte.wmv
Instalando Memcached segunda parte.wmv
::2010/06/20::
Play Video
33
memcached04 PHP操作memcached一
memcached04 PHP操作memcached一
::2014/01/21::
Play Video
34
Can HPC Interconnects Benefit MemCached And Hadoop?
Can HPC Interconnects Benefit MemCached And Hadoop?
::2011/05/03::
Play Video
35
The Garantia Data Memcached Cloud
The Garantia Data Memcached Cloud
::2012/06/10::
Play Video
36
2 Database management   MemCacheD General Usage
2 Database management MemCacheD General Usage
::2013/09/03::
Play Video
37
memcached05 PHP操作memcached二
memcached05 PHP操作memcached二
::2014/01/21::
Play Video
38
Instalando Memcached.wmv
Instalando Memcached.wmv
::2010/06/20::
Play Video
39
PHP advanced caching - godaddy has php 5 5 and memcached module extension
PHP advanced caching - godaddy has php 5 5 and memcached module extension
::2014/03/03::
Play Video
40
How to use Memcached on a Rails app on Heroku
How to use Memcached on a Rails app on Heroku
::2014/02/25::
Play Video
41
5 Database management   MemCacheD Python Integration
5 Database management MemCacheD Python Integration
::2013/09/03::
Play Video
42
High performance RDMA-based design for Big Data and Web 2.0 memcached
High performance RDMA-based design for Big Data and Web 2.0 memcached
::2013/04/24::
Play Video
43
PyCon PL 2013 "Beyond millions of keys in memcached and redis"
PyCon PL 2013 "Beyond millions of keys in memcached and redis"
::2013/11/14::
Play Video
44
memcached06 PHP操作memcached三
memcached06 PHP操作memcached三
::2014/01/21::
Play Video
45
16. Implement Memcached - Question
16. Implement Memcached - Question
::2012/08/10::
Play Video
46
8 Database management   MemCacheD Persistence MemCacheDB
8 Database management MemCacheD Persistence MemCacheDB
::2013/09/03::
Play Video
47
23. Improving Memcache
23. Improving Memcache
::2012/08/28::
Play Video
48
Acceleration for Big Data, Hadoop, and Memcached
Acceleration for Big Data, Hadoop, and Memcached
::2012/03/17::
Play Video
49
Accelerating Big Data with Hadoop (HDFS, MapReduce and HBase) and Memcached
Accelerating Big Data with Hadoop (HDFS, MapReduce and HBase) and Memcached
::2013/03/15::
Play Video
50
memcached07 PHP操作memcached四
memcached07 PHP操作memcached四
::2014/01/21::
NEXT >>
RESULTS [51 .. 101]
From Wikipedia, the free encyclopedia
Jump to: navigation, search
Memcached
Developer(s) Danga Interactive
Initial release May 22, 2003 (2003-05-22)
Stable release 1.4.17 / December 20, 2013 (2013-12-20)[1]
Written in C
Operating system Cross-platform
Type distributed memory caching system
License New BSD License[2]
Website www.memcached.org

In computing, Memcached is a general-purpose distributed memory caching system. It is often used to speed up dynamic database-driven websites by caching data and objects in RAM to reduce the number of times an external data source (such as a database or API) must be read.

Memcached runs on Unix, Linux, Windows and Mac OS X and is distributed under a permissive free software license.[3]

Memcached's APIs provide a giant hash table distributed across multiple machines. When the table is full, subsequent inserts cause older data to be purged in least recently used (LRU) order.[4][5] Applications using Memcached typically layer requests and additions into RAM before falling back on a slower backing store, such as a database.

Memcached was originally developed by Danga Interactive for LiveJournal, but is now used by many other systems, including MocoSpace,[6] YouTube,[7] Reddit,[8] Zynga,[9] Facebook,[10][11][12] Orange,[13] Twitter,[14] Tumblr[15] and Wikipedia.[16] Engine Yard and Jelastic are using Memcached as part of their platform as a service technology stack[17][18] and Heroku offers several Memcached services[19] as part of their platform as a service. Google App Engine, AppScale, Microsoft Azure and Amazon Web Services also offer a Memcached service through an API.[20][21][22][23]

History[edit]

Memcached was first developed by Brad Fitzpatrick for his website LiveJournal, on May 22, 2003.[24][25][26] It was then rewritten in C by Anatoly Vorobey.[27]

Architecture[edit]

The system uses a client–server architecture. The servers maintain a key–value associative array; the clients populate this array and query it. Keys are up to 250 bytes long and values can be at most 1 megabyte in size.

Clients use client-side libraries to contact the servers which, by default, expose their service at port 11211. Each client knows all servers; the servers do not communicate with each other. If a client wishes to set or read the value corresponding to a certain key, the client's library first computes a hash of the key to determine the server to use. Then it contacts that server. The server will compute a second hash of the key to determine where to store or read the corresponding value.

The servers keep the values in RAM; if a server runs out of RAM, it discards the oldest values. Therefore, clients must treat Memcached as a transitory cache; they cannot assume that data stored in Memcached is still there when they need it. MemcacheDB, Couchbase Server, Tarantool and other database servers provide persistent storage while maintaining Memcached protocol compatibility.

If all client libraries use the same hashing algorithm to determine servers, then clients can read each other's cached data.

A typical deployment will have several servers and many clients. However, it is possible to use Memcached on a single computer, acting simultaneously as client and server.

Security[edit]

Most deployments of Memcached exist within trusted networks where clients may freely connect to any server. There are cases, however, where Memcached is deployed in untrusted networks or where administrators would like to exercise control over the clients that are connecting. For this purpose Memcached can be compiled with optional SASL authentication support. The SASL support requires the binary protocol.

A presentation at BlackHat USA 2010 revealed that a number of large public websites had left Memcached open to inspection, analysis, retrieval, and modification of data.[28]

Example code[edit]

Note that all functions described on this page are pseudocode only. Memcached calls and programming languages may vary based on the API used.

Converting database or object creation queries to use Memcached is simple. Typically, when using straight database queries, example code would be as follows:

 function get_foo(int userid) {
    data = db_select("SELECT * FROM users WHERE userid = ?", userid);
    return data;
 }

After conversion to Memcached, the same call might look like the following

 function get_foo(int userid) {
    /* first try the cache */
    data = memcached_fetch("userrow:" + userid);
    if (!data) {
       /* not found : request database */
       data = db_select("SELECT * FROM users WHERE userid = ?", userid);
       /* then store in cache until next get */
       memcached_add("userrow:" + userid, data);
    }
    return data;
 }

The client would first check whether a Memcached value with the unique key "userrow:userid" exists, where userid is some number. If the result does not exist, it would select from the database as usual, and set the unique key using the Memcached API add function call.

However, if only this API call were modified, the server would end up fetching incorrect data following any database update actions: the Memcached "view" of the data would become out of date. Therefore, in addition to creating an "add" call, an update call would also be needed using the Memcached set function.

 function update_foo(int userid, string dbUpdateString) {
   /* first update database */
    result = db_execute(dbUpdateString);
    if (result) {
       /* database update successful : fetch data to be stored in cache */
       data = db_select("SELECT * FROM users WHERE userid = ?", userid);
       /* the previous line could also look like data = createDataFromDBString(dbUpdateString); */
       /* then store in cache until next get */
       memcached_set("userrow:" + userid, data);
    }
 }

This call would update the currently cached data to match the new data in the database, assuming the database query succeeds. An alternative approach would be to invalidate the cache with the Memcached delete function, so that subsequent fetches result in a cache miss. Similar action would need to be taken when database records were deleted, to maintain either a correct or incomplete cache.

See also[edit]

References[edit]

  1. ^ "Release notes for Release 1.4.17". Retrieved 2014-03-07. 
  2. ^ "memcached". Google Code/Memcached Project. Retrieved 2013-05-22. 
  3. ^ "License of Memcached". 
  4. ^ "Memcached NewOverview". 
  5. ^ "Memcached NewUserInternals". 
  6. ^ MocoSpace Architecture - 3 Billion Mobile Page Views a Month. High Scalability (2010-05-03). Retrieved on 2013-09-18.
  7. ^ Cuong Do Cuong (Engineering manager at YouTube/Google) (June 23, 2007). Seattle Conference on Scalability: YouTube Scalability (Online Video - 26th minute). Seattle: Google Tech Talks. 
  8. ^ Steve Huffman on Lessons Learned at Reddit
  9. ^ How Zynga Survived FarmVille
  10. ^ Facebook Developers Resources
  11. ^ Scaling Memcached at Facebook
  12. ^ NSDI '13: Scaling Memcache at Facebook
  13. ^ Orange Developers
  14. ^ It's Not Rocket Science, But It's Our Work
  15. ^ Engineer – Core Applications Group job at Tumblr in New York, NY, powered by JobScore. Jobscore.com. Retrieved on 2013-09-18.
  16. ^ MediaWiki Memcached
  17. ^ Engine Yard Technology Stack
  18. ^ Jelastic Memcached System
  19. ^ Heroku Memcached add-ons
  20. ^ Using Memcache - Google App Engine - Google Code
  21. ^ http://appscale.cs.ucsb.edu Memcached in AppScale
  22. ^ About In-Role Cache for Windows Azure Cache. Msdn.microsoft.com. Retrieved on 2013-09-18.
  23. ^ Amazon ElastiCache. Aws.amazon.com. Retrieved on 2013-09-18.
  24. ^ changelog: livejournal. Community.livejournal.com (2003-05-22). Retrieved on 2013-09-18.
  25. ^ brad's life - weather, running, distributed cache daemon. Brad.livejournal.com (2003-05-22). Retrieved on 2013-09-18.
  26. ^ lj_dev: memcached. Community.livejournal.com (2003-05-27). Retrieved on 2013-09-18.
  27. ^ lj_dev: memcached. Lj-dev.livejournal.com (2003-05-27). Retrieved on 2013-09-18.
  28. ^ BlackHat Write-up: go-derper and mining memcaches
  29. ^ "Speedy MySQL 5.6 takes aim at NoSQL, MariaDB."
  30. ^ Tarantool user guide

External links[edit]

Commercially Supported Distributions[edit]

Wikipedia content is licensed under the GFDL License

Mashpedia enables any individual or company to promote their own Youtube-hosted videos or Youtube Channels, offering a simple and effective plan to get them in front of our engaged audience.

Want to learn more? Please contact us at: hello@mashpedia.com

Powered by YouTube
LEGAL
  • Mashpedia © 2014