1.Getting Started: Redis and Python

So if you have been following NoSQL movement, the migration of some types of data to non-relational datastores has recently picked up speed. For web (and other developers) this has lead to some impressive engineering resources developing some amazing tools being open sourced for the world to use. One that caught my eye recently has been Salvatore Sanfilippo’s redis which has been taken under the wing of VMWare, solidifying and validating the great work that Salavatore has done making redis an amazing tool in any developers arsenal.

A very simplified explanation of redis is that it is an in memory key-value store like memcached but, it is persistent on disk, unlike memcached which is volatile. Along with being disk-persistent redis also supports some basic data structures like lists, sets, ordered sets, hashes, and of course basic key-value string storage like memcached. Redis, with disk-persistence and basic data structures, remains blazing fast with published benchmarks of 110,000 SETs per second, about 81,000 GETs per second.

This post is the start of a series of articles on redis for Python programmers. A prerequisite for this is going to be some basic Python knowledge, which if you haven’t used before I highly recommend the free web book Dive Into Python. This is going to be a simple overview of the basic data types and usage for Python programmers and we will slowly progress into more complex usages, so if you haven’t done anything with redis before this is a perfect start.

Installing Redis

Redis is extremely easy to install on POSIX systems (Linux, OSX, Cygwin for Windows). Simply download either a release version or checkout the trunk version of redis from the project home page. Unpackage, make, and run. That’s it!

 

  1. gettingstarted $ wget http://redis.googlecode.com/files/redis-1.2.5.tar.gz
  2. gettingstarted $ gunzip redis-1.2.5.tar.gz
  3. gettingstarted $ tar -xvf redis-1.2.5.tar
  4. gettingstarted $ cd redis-1.2.5
  5. redis-1.2.5 $ make
  6. redis-1.2.5 $ ./redis-server
  7. 21 Mar 17:37:11 * Warning: no config file specified, using the default config. In order to specify a config file use 'redis-server /path/to/redis.conf'
  8. 21 Mar 17:37:11 - Server started, Redis version 1.2.5
  9. 21 Mar 17:37:11 - The server is now ready to accept connections on port 6379
  10. 21 Mar 17:37:11 . 0 clients connected (0 slaves), 832496 bytes in use, 0 shared objects

6 commands and redis is running and ready for a connection. Obviously this isn’t best practice for running redis in production but, it works well for just getting it up and running for some experimentation. Let’s leave that window open and open another for a Python console.

In every example below we only are covering the basic operations of each type. If you want to learn about all of the possible operations for redis read up on the redis Command Reference. This has all of the commands available to you.

Redis and Python

First thing we need is the redis-py module for Python. You can install this manually or just use easy_install.

  1. easy_install redis
Everything has a place: keys

Redis is a key-value store just like a Python dictionary. So everything that goes into redis no matter what it is, has a key (or location, or address, whatever makes sense to you). You can’t store something without a place for it to go

Now let’s open up a Python console and start to play with the basic data structures

Data structure: String

The most basic data structure in redis is the string. Simply, you have a key and it stores a string.

 

  1. >>> import redis
  2. >>> r_server = redis.Redis("localhost")
  3. >>> r_server.set("name", "DeGizmo")
  4. True
  5. >>> r_server.get("name")
  6. 'DeGizmo'

So, line by line let’s walk through what we’ve done here.

  1. import the redis module
  2. create an object called “r_server” of redis.Redis type with the parameter “localhost” which is the network location of our running redis server
  3. call the function “set” on the “r_server” object with the parameters “name” and “DeGizmo”. The first parameter “name” is the key and “DeGizmo” is the value. So in redis we created a key-value pair of “name” -> “DeGizmo”
  4. call the function “get” which will retrieve the value from the key which is passed as the parameter. We are getting the key “name” and redis returns the value “DeGizmo”

That’s it. If you go and stop the redis server and restart it and run r_server.get(“name”) command again, it will return “DeGizmo” again due to redis’s persistent store.

Data structure: Integer

The second data type available is an integer.

 

  1. >>> r_server.set("hit_counter", 1)
  2. True
  3. >>> r_server.incr("hit_counter")
  4. 2
  5. >>> r_server.get("hit_counter")
  6. '2'
  7. >>> r_server.decr("hit_counter")
  8. 1
  9. >>> r_server.get("hit_counter")
  10. '1'

Again using the “r_server” Redis object we run the following commands

  1. We set the redis key “hit_counter” to a value of 1 just like in the example above
  2. Then we run the command “incr” on the key “hit_counter”. This will increment (incr) the key by 1
  3. We get the value for the key “hit_counter” to verify that it was been incremented
  4. Then we run the command “decr” on the key “hit_counter”. This will decrement (decr) the key by 1
  5. We get the value for the key “hit_counter” to verify that it was been decremented

Simple huh? Well remember that redis is a persistent store that can be accessed by multiple machines (or programs) simultaneously. If you have a horizontally scaled web farm with many different web servers running on separate machines, your web app can all call on and reference the same redis instance and all operate on the “hit_counter” key at the same time and increment it just like how you would use a MySQL database. You can kind of think of redis like a “shared” storage across different applications.

Data structure: Lists

Very similar to the built in Python list type, the redis list type has a few basic methods that combined can quite powerful. We are only covering a tiny portion of the commands available, you can find all the commands in the redis Command Reference.

 

  1. >>> r_server.rpush("members", "Adam")
  2. True
  3. >>> r_server.rpush("members", "Bob")
  4. True
  5. >>> r_server.rpush("members", "Carol")
  6. True
  7. >>> r_server.lrange("members", 0, -1)
  8. ['Adam', 'Bob', 'Carol']
  9. >>> r_server.llen("members")
  10. 3
  11. >>> r_server.lindex("members", 1)
  12. 'Bob'

We don’t actually have to create or declare anything when using redis, we just use it and redis handles everything else for us. So we just go ahead and tell redis at the key “members” we are going to add (rpush, right push) the value “Adam”. Redis creates the list for us at key “members” and adds the value “Adam” to the list.

  1. With the r_server object again we call the method “rpush” which will add the value “Adam” to the newly created list “members”
  2. We add “Bob” to the same list
  3. Finally we’ll add “Carol”
  4. With the lrange method we are asking redis to return all the objects in “members”. lrange takes 3 arguments: key, start index in list, end index in list. We are getting the objects from the key “members” starting at index 0 and ending at -1 (which is technically the -1, or last index in the list, which will return everything)
  5. The llen method asks redis to return the length of the list at the key “members” which now has 3 objects
  6. lindex method tells redis that we want the object from the key “members” at the index position of 1 (remember lists start at index 0), so redis returns “Bob”

We’ve got some elements in the list at the key “members”; now lets get remove some elements.

 

  1. >>> r_server.rpop("members")
  2. 'Carol'
  3. >>> r_server.lrange("members", 0, -1)
  4. ['Adam', 'Bob']
  5. >>> r_server.lpop("members")
  6. 'Adam'
  7. >>> r_server.lrange("members", 0, -1)
  8. ['Bob']
  1. With the method rpop (right pop) we remove the element in the list on the right side (tail), which is “Carol”
  2. Now when we ask for the list “members” from redis again (from the start of 0, and the end of -1 which returns everything) we see our list now doesn’t have “Carol”
  3. We now lpop (left pop) an element from the list “members”. This will remove the far left element from the list, “Adam”
  4. Now the entire list only contains “Bob”
Data structure: Set

Again, sets perform identically to the built in Python set type. Simply, sets are lists but, can only have unique values. In the above example if we added the value Adam (r_server.lpush(“members”, “Adam”) ) 20 times our list would contain 20 values all containing the value “Adam”. In a set, all elements are unique.

 

  1. >>> r_server.delete("members")
  2. True
  3. >>> r_server.sadd("members", "Adam")
  4. True
  5. >>> r_server.sadd("members", "Bob")
  6. True
  7. >>> r_server.sadd("members", "Carol")
  8. True
  9. >>> r_server.sadd("members", "Adam")
  10. False
  11. >>> r_server.smembers("members")
  12. set(['Bob', 'Adam', 'Carol'])
  1. First off we delete the old list in the key “members” so we can use it as a set
  2. Then we sadd (set add) the value “Adam” to the key “members”
  3. Do the same for the value “Bob”
  4. Do the same for the value “Carol”
  5. Now we try to add the value “Adam” to the key “members” again but, this time it returns “False” since we are working on a set, and a set only has unique values. There already is a value “Adam” present in this set
  6. The method smembers returns all the members of the set in the Python type set

An example of a use of sets in a web application would be for “upvotes” on a reddit, or hacker news type website. We want to keep track of who up votes a story but, you should only be able to up vote a story once.

 

  1. >>> r_server.sadd("story:5419:upvotes", "userid:9102")
  2. True
  3. >>> r_server.sadd("story:5419:upvotes", "userid:12981")
  4. True
  5. >>> r_server.sadd("story:5419:upvotes", "userid:1233")
  6. True
  7. >>> r_server.sadd("story:5419:upvotes", "userid:9102")
  8. False
  9. >>> r_server.scard("story:5419:upvotes")
  10. 3
  11. >>> r_server.smembers("story:5419:upvotes")
  12. set(['userid:12981', 'userid:1233', 'userid:9102'])

 

I added a little twist in here with the name of the key: “story:5419:upvotes” but, it’s easy to explain. Redis is “flat” with it’s keyspace. So if we have many different stories we use a fixed key naming convention for easy reference in redis. For this example our key is broken down like this: “object type” : “id” : “attribute”. So, we have an object of type “story” with an ID of “5419″ with an attribute “upvotes’. Redis has no idea what any of this means it just knows the key is “story:5419:upvotes” but, it doesn’t matter, we know what it means and we can divide up our objects into this name space to make it easier to work with and keep from “losing” things. The value being added to the key is divided up in the same way. “userid” is the type and “9102″ is the value or the ID for that user voting on the story.

  1. Just like before we are adding the value “userid:9102″ to the key “story:5419:upvotes”
  2. Now we are adding the value “userid:12981″
  3. Finally adding the valud “userid:1233″
  4. Now, the user with the ID 9102 tries to up vote the story with the ID 5419 again, and redis returns False since that user has already up votes this story before and you can’t up vote a story twice!
  5. The method “scard” is asking redis for the cardinality of the set at key “story:5419:upvotes” or how many things are in this set, and redis returns 3.
  6. Finally we return the list of userid’s that we have stored in the set.
Data structure: Ordered sets

The last data structure we are going to talk about today is an ordered (or sorted) set. This operates just like a set but, has an extra attribute when we add something to a set called a “score”. This score determines the order of the elements in the set. We will stick with the concept for this final example

 

  1. >>> r_server.zadd("stories:frontpage", "storyid:3123", 34)
  2. True
  3. >>> r_server.zadd("stories:frontpage", "storyid:9001", 3)
  4. True
  5. >>> r_server.zadd("stories:frontpage", "storyid:2134", 127)
  6. True
  7. >>> r_server.zadd("stories:frontpage", "storyid:2134", 127)
  8. False
  9. >>> r_server.zrange("stories:frontpage", 0, -1, withscores=True)
  10. [('storyid:9001', 3.0), ('storyid:3123', 34.0), ('storyid:2134', 127.0)]
  11. >>> frontpage = r_server.zrange("stories:frontpage", 0, -1, withscores=True)
  12. >>> frontpage.reverse()
  13. >>> frontpage
  14. [('storyid:2134', 127.0), ('storyid:3123', 34.0), ('storyid:9001', 3.0)]

Quick namespace explanation like before. For the key we are going to be referring to “stories:frontpage” which is going to be a set of stories slated for the front page of our website. We are storing in that key the value of “storyid:3123″ which is the ID of some story on the site and then a score, which in our case is going to be the number of votes on a story.

  1. First we add the value “storyid:3123″ to “stories:frontpage”, and “storyid:3123″ in our example is going to have 34 votes.
  2. Then add “storyid:9001″ with 3 votes
  3. Then add “storyid:2134″ with 127 votes
  4. We are going to try to add “story:2134″ to the set again but, we can’t since it already exists.
  5. Now we are going to ask redis for all the elements in “stories:frontpage” from index 0 to index -1 (the end of the list) with all associated scores (withscores=True)
  6. We’ve got the scores but, they are in ascending order, we want them in descending order for our website, so we are going to store the results in the variable “frontpage”
  7. Then reverse it (which is an in place operation in Python)
  8. Now print out the front page!

In conclusion let’s do a quick example of a “view” in an application in which a user will vote of a story using redis as a storage engine

 

  1. #given variables

  2. #r_server = our redis server

  3. #user_id = the user who voted on the story

  4. #story_id = the story which the user voted on

  5. if r_server.sadd("story:%s"% story_id,"userid:%s"% user_id):

  6.      r_server.zincrby("stories:frontpage","storyid:%s"% story_id, 1)

2 lines of code? This is might compact but, once we unravel it we can see how it makes sense and how powerful redis can be. Let’s start with the if statement.

  1. if r_server.sadd("story:%s"% story_id,"userid:%s"% user_id):

We know the command “sadd” already. This will add an element to a set at a key. The key in this case is

 

  1. "story:%s"% story_id

If story_id is 3211, then the resulting string will be “story:3211″. This is the key in redis which contains the list of users that has voted on the story.

The value to be inserted at this key is

 

  1. "userid:%s"% user_id

Just like with story, if the user_id is 9481 then the string to be inserted into the set at “story:3211″ will be “user_id:9481″

Now the redis command “sadd” will return False if that element is already present in the set. So if a user has already voted on this story before we don’t execute the statement under the if. But, if it is added, then we have a brand new vote and we have to increment the votes for the front page.

 

  1. r_server.zincrby("stories:frontpage","storyid:%s"% story_id, 1)

 

We have an ordered set at the key “stories:frontpage” and we are going to increment the element “storyid:%s” % story_id (“story:3211″) by 1.

And now we’re done! We’ve made sure the user hasn’t voted on this story before and then we’ve incremented the number of votes for this story on the front page!

文章來自:http://degizmo.com/2010/03/22/getting-started-redis-and-python/

更多相關文章
  • Redis資料彙總專題 作者:nosqlfan  很多朋友反映,說NoSQLFan上的資料不少,但是要找到自己實用的太難,于是萌生做這樣一個專題的想法.通過將不同NoSQL産品從入門到精通的各種資料進行彙總,希望能夠讓大家更快的找到適合自己的教程或文章進行閱讀. 本篇爲Redis部分的資料彙總.保持 ...
  • 1. Data layout     當我們使用redis時候,我們並沒有表.所以我們需要key來標識對象,而value來存儲所需要的值.對于retwis(twitter的簡易版本)來說,我們可以通過唯一的ID來標識用戶,通過INCR操作即可: 127.0.0.1:6379> set next ...
  • 1.Redis是什麽? 十五分鍾介紹 Redis數據結構 Redis系統性介紹 一個很棒的Redis介紹PPT 強烈推薦!非同一般的Redis介紹 Redis之七種武器 鋒利的Redis redis 適用場景與實現 [翻譯]Redis協定 2.Redis內部實現 Redis源碼分析系列文章 Redi ...
  • 12.python通過redis-py,redisco操作redis
    1.安裝redis 1.1.准備部分: 什麽是redis? Redis是REmote DIctionary Server的縮寫.一個非關系型數據庫 爲什麽要用redis? 1, 因爲redis快,Redis is pretty fast!, 110000 SETs/second, 81000 GET ...
  • #!/usr/bin/python #coding=utf-8 import redis class urlList(): ############################################# # create by myjack                         ...
  • #!/usr/env/bin python import redis r=redis.Redis(host='localhost',port=6379,db=0) with r.pipeline() as pipe:     while 1:         try:             pip ...
  • #1.安裝 好吧,我承認我只會最簡單的安裝 sudo apt-get install redis-server python 支持包: (其實就一個文件,搞過來就能用) sudo apt-get install python-redis #2.配置 配置一下吧,默認配置文件在: "/etc ...
  • 1. 安裝pyredis 首先安裝pip # apt-get install python-pip ...... # pip install --proxy=http://172.1.2.6:8080 redis   Downloading redis-2.9.1.tar.gz (62kB): 62 ...
一周排行
  • JMS API
    JMS是Sun公司通過JSR-914專案創建的一種企業消息傳送API.JMS自身並不是一種 ...
  • 華爲公司 java 面試題   第一部分:選擇題 QUESTION NO: 1 1.public class Test {     public static void changeStr(String str){ ...
  • 1.基本動作     Cocos2d提供的基本動作:瞬時動作.延時動作.運作速度.     瞬時動作:就是不需要時間,馬上就完成的動作.瞬時動作的共同基類是 InstantAction.     Cocos2d提供以 ...
  •  Web前端   最標准化的3項技術:HTML.JavaScript.CSS.   其中,HTML主要使用4,JS框架主要使用jquery,CSS框架主要使用Bootstrap.  好處: 簡單易學,沒有什麽學習成本 ...
  • easyui datagrid 慢
    這個標題估計使用 easyui 的同學都搜過吧,我也搜索.糾結了很長時間,今天在詳細閱讀 ...
  • 文章refer to: http://www.cnblogs.com/lwm-1988/archive/2011/08/20/2147299.html     有兩種方式:   1. command & : ...
  • 100offer : http://www.100offer.com/i/gywhvsxl
  • Linux顯示時間戳:  date +%s   #顯示時間戳 1435798986   顯示指定時間的時間戳:date -d "0 0" +%s    127959 ...
  • Address Bar
    Address Bar 實現了類似 iOS Safari 浏覽器地址欄的控件. 轉載:ht ...
  • 16個最棒的WordPress婚紗攝影網站主題
    今天我們整理了16個最棒的WordPress婚紗攝影網站主題,希望大家喜歡. 原文地址:h ...