《Python进阶》学习笔记
1. *args 和 **kwargs
*args 是用来发送一定长度的键值对, 作为参数传递给一个函数。 **kwargs 允许你将不定长度的键值对, 作为参数传递给⼀个函数。
2. 调试 Debugging
命令行使用Python debugger运行一个脚本: $ python -m pdb my_script.py pdb模块(断点调试)用法: import pdb 功能:断点设置,单步执行,查看代码,查看函数,追踪变量等 pdb.set_trace() 功能 : 设置初始断点,开始进入pdb调试模式调试中的命令 break , b : 设置断点 continue ,c : 继续执行 w , : 显示当前正在执行的代码行的上下文信息 list , l : 查看当前代码段 next, n : 单步执行 step, s : 进入函数单步执行 pp : 打印变量值 help : 帮助 q : 退出调试 exit : 退出pdb调试
3.set 数据结构
交集:你可以对比两个集合的交集(两个集合中都有的数据) set_a.intersection(set_b) : set_a和set_b共同拥有的元素,以set形式返回; 差集:你可以用差集(difference)找出无效的数据,相当于用一个集合减去另一个集合的数据 set_a.difference(set_b) : set_a减去set_b,相当于set_a单独拥有的元素,以set形式返回; 也可以用{}符号来创建集合,如: a_set = {'red', 'blue', 'green'} print(type(a_set)) 输出: <type 'set'>
4. slots魔法方法
slots来告诉Python,当一个类需要创建大量实例时,可以通过slots声明实例所需要的属性,不要使用字典,而且只给一个固定集合的属性分配空间。 这样做带来以下优点:
- 更快的属性访问速度
- 减少内存消耗
- 利用这种特性来限制实例的属性。
默认情况下,访问一个实例的属性是通过访问该实例的dict来实现的。如访问a.x就相当于访问a.dict['x']。为了便于理解,我粗略地将它拆分为四步: a.x 2. a.dict 3. a.dict['x'] 4. 结果
从slots的实现可以得知,定义了slots的类会为每个属性创建一个描述器。访问属性时就直接调用这个描述器。在这里我将它拆分为三步: b.x 2. member decriptor 3. 结果
我在上文提到,访问dict和描述器的速度是相近的,而通过dict访问属性多了a.dict['x']字典访值一步(一个哈希函数的消耗)。由此可以推断出,使用了slots的类的属性访问速度比没有使用的要快。 [参考文章:Pythonslots详解 - rainfd - 博客园 https://www.cnblogs.com/rainfd/p/slots.html]
5.容器数据类型 Collections
重点: 5.1 defaultdict :字典的子类,提供了一个工厂函数,为字典查询提供一个默认值; 文档:
Help on class defaultdict in module collections:
class defaultdict(builtins.dict)
| defaultdict(default_factory[, ...]) --> dict with default factory
|
| The default factory is called without arguments to produce
| a new value when a key is not present, in __getitem__ only.
| A defaultdict compares equal to a dict with the same items.
| All remaining arguments are treated the same as if they were
| passed to the dict constructor, including keyword arguments.
|
| Method resolution order:
| defaultdict
| builtins.dict
| builtins.object
|
| Methods defined here:
|
| __copy__(...)
| D.copy() -> a shallow copy of D.
|
| __getattribute__(self, name, /)
| Return getattr(self, name).
|
| __init__(self, /, *args, **kwargs)
| Initialize self. See help(type(self)) for accurate signature.
|
| __missing__(...)
| __missing__(key) # Called by __getitem__ for missing key; pseudo-code:
| if self.default_factory is None: raise KeyError((key,))
| self[key] = value = self.default_factory()
| return value
|
| __reduce__(...)
| Return state information for pickling.
|
| __repr__(self, /)
| Return repr(self).
|
| copy(...)
| D.copy() -> a shallow copy of D.
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| default_factory
| Factory for default value called by __missing__().
|
| ----------------------------------------------------------------------
| Methods inherited from builtins.dict:
|
| __contains__(self, key, /)
| True if the dictionary has the specified key, else False.
|
| __delitem__(self, key, /)
| Delete self[key].
|
| __eq__(self, value, /)
| Return self==value.
|
| __ge__(self, value, /)
| Return self>=value.
|
| __getitem__(...)
| x.__getitem__(y) <==> x[y]
|
| __gt__(self, value, /)
| Return self>value.
|
| __iter__(self, /)
| Implement iter(self).
|
| __le__(self, value, /)
| Return self<=value.
|
| __len__(self, /)
| Return len(self).
|
| __lt__(self, value, /)
| Return self<value.
|
| __ne__(self, value, /)
| Return self!=value.
|
| __setitem__(self, key, value, /)
| Set self[key] to value.
|
| __sizeof__(...)
| D.__sizeof__() -> size of D in memory, in bytes
|
| clear(...)
| D.clear() -> None. Remove all items from D.
|
| get(self, key, default=None, /)
| Return the value for key if key is in the dictionary, else default.
|
| items(...)
| D.items() -> a set-like object providing a view on D's items
|
| keys(...)
| D.keys() -> a set-like object providing a view on D's keys
|
| pop(...)
| D.pop(k[,d]) -> v, remove specified key and return the corresponding value.
| If key is not found, d is returned if given, otherwise KeyError is raised
|
| popitem(...)
| D.popitem() -> (k, v), remove and return some (key, value) pair as a
| 2-tuple; but raise KeyError if D is empty.
|
| setdefault(self, key, default=None, /)
| Insert key with a value of default if key is not in the dictionary.
|
| Return the value for key if key is in the dictionary, else default.
|
| update(...)
| D.update([E, ]**F) -> None. Update D from dict/iterable E and F.
| If E is present and has a .keys() method, then does: for k in E: D[k] = E[k]
| If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v
| In either case, this is followed by: for k in F: D[k] = F[k]
|
| values(...)
| D.values() -> an object providing a view on D's values
|
| ----------------------------------------------------------------------
| Class methods inherited from builtins.dict:
|
| fromkeys(iterable, value=None, /) from builtins.type
| Create a new dictionary with keys from iterable and values set to value.
|
| ----------------------------------------------------------------------
| Static methods inherited from builtins.dict:
|
| __new__(*args, **kwargs) from builtins.type
| Create and return a new object. See help(type) for accurate signature.
|
| ----------------------------------------------------------------------
| Data and other attributes inherited from builtins.dict:
|
| __hash__ = None
与一般dict区别:标准字典包括setdefault方法()获取一个值,如果值不存在,建立一个默认; 相比之下,defaultdict允许调用者在初始化时预先设置默认值。
>>> from collections import defaultdict
>>> dd = defaultdict(lambda: 'N/A')
>>> dd['key1'] = 'abc'
>>> dd['key1'] # key1存在
'abc'
>>> dd['key2'] # key2不存在,返回默认值
'N/A'
使用dict时,如果引用的Key不存在,就会抛出KeyError。如果希望key不存在时,返回一个默认值,就可以用defaultdict 注意:默认值是调用函数返回的,而函数在创建defaultdict对象时传入。
5.2 Counter:字典的子类,提供了可哈希对象的计数功能
文档:
class Counter(builtins.dict)
| Counter(*args, **kwds)
|
| Dict subclass for counting hashable items. Sometimes called a bag
| or multiset. Elements are stored as dictionary keys and their counts
| are stored as dictionary values.
|
| >>> c = Counter('abcdeabcdabcaba') # count elements from a string
|
| >>> c.most_common(3) # three most common elements
| [('a', 5), ('b', 4), ('c', 3)]
| >>> sorted(c) # list all unique elements
| ['a', 'b', 'c', 'd', 'e']
| >>> ''.join(sorted(c.elements())) # list elements with repetitions
| 'aaaaabbbbcccdde'
| >>> sum(c.values()) # total of all counts
| 15
|
| >>> c['a'] # count of letter 'a'
| 5
| >>> for elem in 'shazam': # update counts from an iterable
| ... c[elem] += 1 # by adding 1 to each element's count
| >>> c['a'] # now there are seven 'a'
| 7
| >>> del c['b'] # remove all 'b'
| >>> c['b'] # now there are zero 'b'
| 0
|
| >>> d = Counter('simsalabim') # make another counter
| >>> c.update(d) # add in the second counter
| >>> c['a'] # now there are nine 'a'
| 9
|
| >>> c.clear() # empty the counter
| >>> c
| Counter()
|
| Note: If a count is set to zero or reduced to zero, it will remain
| in the counter until the entry is deleted or the counter is cleared:
|
| >>> c = Counter('aaabbc')
| >>> c['b'] -= 2 # reduce the count of 'b' by two
| >>> c.most_common() # 'b' is still in, but its count is zero
| [('a', 3), ('c', 1), ('b', 0)]
|
| Method resolution order:
| Counter
| builtins.dict
| builtins.object
|
| Methods defined here:
|
| __add__(self, other)
| Add counts from two counters.
|
| >>> Counter('abbb') + Counter('bcc')
| Counter({'b': 4, 'c': 2, 'a': 1})
|
| __and__(self, other)
| Intersection is the minimum of corresponding counts.
|
| >>> Counter('abbb') & Counter('bcc')
| Counter({'b': 1})
|
| __delitem__(self, elem)
| Like dict.__delitem__() but does not raise KeyError for missing values.
|
| __iadd__(self, other)
| Inplace add from another counter, keeping only positive counts.
|
| >>> c = Counter('abbb')
| >>> c += Counter('bcc')
| >>> c
| Counter({'b': 4, 'c': 2, 'a': 1})
|
| __iand__(self, other)
| Inplace intersection is the minimum of corresponding counts.
|
| >>> c = Counter('abbb')
| >>> c &= Counter('bcc')
| >>> c
| Counter({'b': 1})
|
| __init__(*args, **kwds)
| Create a new, empty Counter object. And if given, count elements
| from an input iterable. Or, initialize the count from another mapping
| of elements to their counts.
|
| >>> c = Counter() # a new, empty counter
| >>> c = Counter('gallahad') # a new counter from an iterable
| >>> c = Counter({'a': 4, 'b': 2}) # a new counter from a mapping
| >>> c = Counter(a=4, b=2) # a new counter from keyword args
|
| __ior__(self, other)
| Inplace union is the maximum of value from either counter.
|
| >>> c = Counter('abbb')
| >>> c |= Counter('bcc')
| >>> c
| Counter({'b': 3, 'c': 2, 'a': 1})
|
| __isub__(self, other)
| Inplace subtract counter, but keep only results with positive counts.
|
| >>> c = Counter('abbbc')
| >>> c -= Counter('bccd')
| >>> c
| Counter({'b': 2, 'a': 1})
|
| __missing__(self, key)
| The count of elements not in the Counter is zero.
|
| __neg__(self)
| Subtracts from an empty counter. Strips positive and zero counts,
| and flips the sign on negative counts.
|
| __or__(self, other)
| Union is the maximum of value in either of the input counters.
|
| >>> Counter('abbb') | Counter('bcc')
| Counter({'b': 3, 'c': 2, 'a': 1})
|
| __pos__(self)
| Adds an empty counter, effectively stripping negative and zero counts
|
| __reduce__(self)
| Helper for pickle.
|
| __repr__(self)
| Return repr(self).
|
| __sub__(self, other)
| Subtract count, but keep only results with positive counts.
|
| >>> Counter('abbbc') - Counter('bccd')
| Counter({'b': 2, 'a': 1})
|
| copy(self)
| Return a shallow copy.
|
| elements(self)
| Iterator over elements repeating each as many times as its count.
|
| >>> c = Counter('ABCABC')
| >>> sorted(c.elements())
| ['A', 'A', 'B', 'B', 'C', 'C']
|
| # Knuth's example for prime factors of 1836: 2**2 * 3**3 * 17**1
| >>> prime_factors = Counter({2: 2, 3: 3, 17: 1})
| >>> product = 1
| >>> for factor in prime_factors.elements(): # loop over factors
| ... product *= factor # and multiply them
| >>> product
| 1836
|
| Note, if an element's count has been set to zero or is a negative
| number, elements() will ignore it.
|
| most_common(self, n=None)
| List the n most common elements and their counts from the most
| common to the least. If n is None, then list all element counts.
|
| >>> Counter('abcdeabcdabcaba').most_common(3)
| [('a', 5), ('b', 4), ('c', 3)]
|
| subtract(*args, **kwds)
| Like dict.update() but subtracts counts instead of replacing them.
| Counts can be reduced below zero. Both the inputs and outputs are
| allowed to contain zero and negative counts.
|
| Source can be an iterable, a dictionary, or another Counter instance.
|
| >>> c = Counter('which')
| >>> c.subtract('witch') # subtract elements from another iterable
| >>> c.subtract(Counter('watch')) # subtract elements from another counter
| >>> c['h'] # 2 in which, minus 1 in witch, minus 1 in watch
| 0
| >>> c['w'] # 1 in which, minus 1 in witch, minus 1 in watch
| -1
|
| update(*args, **kwds)
| Like dict.update() but add counts instead of replacing them.
|
| Source can be an iterable, a dictionary, or another Counter instance.
|
| >>> c = Counter('which')
| >>> c.update('witch') # add elements from another iterable
| >>> d = Counter('watch')
| >>> c.update(d) # add elements from another counter
| >>> c['h'] # four 'h' in which, witch, and watch
| 4
|
| ----------------------------------------------------------------------
| Class methods defined here:
|
| fromkeys(iterable, v=None) from builtins.type
| Create a new dictionary with keys from iterable and values set to value.
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)
|
| ----------------------------------------------------------------------
| Methods inherited from builtins.dict:
|
| __contains__(self, key, /)
| True if the dictionary has the specified key, else False.
|
| __eq__(self, value, /)
| Return self==value.
|
| __ge__(self, value, /)
| Return self>=value.
|
| __getattribute__(self, name, /)
| Return getattr(self, name).
|
| __getitem__(...)
| x.__getitem__(y) <==> x[y]
|
| __gt__(self, value, /)
| Return self>value.
|
| __iter__(self, /)
| Implement iter(self).
|
| __le__(self, value, /)
| Return self<=value.
|
| __len__(self, /)
| Return len(self).
|
| __lt__(self, value, /)
| Return self<value.
|
| __ne__(self, value, /)
| Return self!=value.
|
| __setitem__(self, key, value, /)
| Set self[key] to value.
|
| __sizeof__(...)
| D.__sizeof__() -> size of D in memory, in bytes
|
| clear(...)
| D.clear() -> None. Remove all items from D.
|
| get(self, key, default=None, /)
| Return the value for key if key is in the dictionary, else default.
|
| items(...)
| D.items() -> a set-like object providing a view on D's items
|
| keys(...)
| D.keys() -> a set-like object providing a view on D's keys
|
| pop(...)
| D.pop(k[,d]) -> v, remove specified key and return the corresponding value.
| If key is not found, d is returned if given, otherwise KeyError is raised
|
| popitem(...)
| D.popitem() -> (k, v), remove and return some (key, value) pair as a
| 2-tuple; but raise KeyError if D is empty.
|
| setdefault(self, key, default=None, /)
| Insert key with a value of default if key is not in the dictionary.
|
| Return the value for key if key is in the dictionary, else default.
|
| values(...)
| D.values() -> an object providing a view on D's values
|
| ----------------------------------------------------------------------
| Static methods inherited from builtins.dict:
|
| __new__(*args, **kwargs) from builtins.type
| Create and return a new object. See help(type) for accurate signature.
|
| ----------------------------------------------------------------------
| Data and other attributes inherited from builtins.dict:
|
| __hash__ = None
from collections import Counter
Counter是一个简单的计数器,例如,统计字符出现的个数,空的Counter容器可以无参数构造,并采用update()方法进行更新;
most_common()返回前n个最多的数据,counter.most_common(x)返回前x个最多的数据;
Counter实例支持聚合结果的算术和集合操作;
elements()方法可以返回一个包含所有Counter数据的迭代器,counter.elements()。
5.3 deque 双端队列 :
文档:
class deque(builtins.object)
| deque([iterable[, maxlen]]) --> deque object
|
| A list-like sequence optimized for data accesses near its endpoints.
|
| Methods defined here:
|
| __add__(self, value, /)
| Return self+value.
|
| __bool__(self, /)
| self != 0
|
| __contains__(self, key, /)
| Return key in self.
|
| __copy__(...)
| Return a shallow copy of a deque.
|
| __delitem__(self, key, /)
| Delete self[key].
|
| __eq__(self, value, /)
| Return self==value.
|
| __ge__(self, value, /)
| Return self>=value.
|
| __getattribute__(self, name, /)
| Return getattr(self, name).
|
| __getitem__(self, key, /)
| Return self[key].
|
| __gt__(self, value, /)
| Return self>value.
|
| __iadd__(self, value, /)
| Implement self+=value.
|
| __imul__(self, value, /)
| Implement self*=value.
|
| __init__(self, /, *args, **kwargs)
| Initialize self. See help(type(self)) for accurate signature.
|
| __iter__(self, /)
| Implement iter(self).
|
| __le__(self, value, /)
| Return self<=value.
|
| __len__(self, /)
| Return len(self).
|
| __lt__(self, value, /)
| Return self<value.
|
| __mul__(self, value, /)
| Return self*value.
|
| __ne__(self, value, /)
| Return self!=value.
|
| __reduce__(...)
| Return state information for pickling.
|
| __repr__(self, /)
| Return repr(self).
|
| __reversed__(...)
| D.__reversed__() -- return a reverse iterator over the deque
|
| __rmul__(self, value, /)
| Return value*self.
|
| __setitem__(self, key, value, /)
| Set self[key] to value.
|
| __sizeof__(...)
| D.__sizeof__() -- size of D in memory, in bytes
|
| append(...)
| Add an element to the right side of the deque.
|
| appendleft(...)
| Add an element to the left side of the deque.
|
| clear(...)
| Remove all elements from the deque.
|
| copy(...)
| Return a shallow copy of a deque.
|
| count(...)
| D.count(value) -> integer -- return number of occurrences of value
|
| extend(...)
| Extend the right side of the deque with elements from the iterable
|
| extendleft(...)
| Extend the left side of the deque with elements from the iterable
|
| index(...)
| D.index(value, [start, [stop]]) -> integer -- return first index of value.
| Raises ValueError if the value is not present.
|
| insert(...)
| D.insert(index, object) -- insert object before index
|
| pop(...)
| Remove and return the rightmost element.
|
| popleft(...)
| Remove and return the leftmost element.
|
| remove(...)
| D.remove(value) -- remove first occurrence of value.
|
| reverse(...)
| D.reverse() -- reverse *IN PLACE*
|
| rotate(...)
| Rotate the deque n steps to the right (default n=1). If n is negative, rotates left.
|
| ----------------------------------------------------------------------
| Static methods defined here:
|
| __new__(*args, **kwargs) from builtins.type
| Create and return a new object. See help(type) for accurate signature.
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| maxlen
| maximum size of a deque or None if unbounded
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| __hash__ = None
类似列表(list)的容器,实现了在两端快速添加(append)和弹出(pop),deque是为了高效实现插入和删除操作的双向列表,适合用于队列和栈, deque除了实现list的append()和pop()外,还支持左端插入appendleft()和左端删除popleft(),这样就可以非常高效地往头部添加或 删除元素。 由于双端队列是线程安全的,在单独的线程中内容甚至可以从两端同时消费。
5.4 namedtuple 命名元组 :创建命名元组子类的工厂函数
文档:
Help on function namedtuple in module collections:
namedtuple(typename, field_names, *, rename=False, defaults=None, module=None)
Returns a new subclass of tuple with named fields.
>>> Point = namedtuple('Point', ['x', 'y'])
>>> Point.__doc__ # docstring for the new class
'Point(x, y)'
>>> p = Point(11, y=22) # instantiate with positional args or keywords >>> p[0] + p[1] # indexable
like a plain tuple
33
>>> x, y = p # unpack like a regular tuple
>>> x, y
(11, 22)
>>> p.x + p.y # fields also accessible by name
33
>>> d = p._asdict() # convert to a dictionary
>>> d['x']
11
>>> Point(**d) # convert from a dictionary
Point(x=11, y=22)
>>> p._replace(x=100) # _replace() is like str.replace() but targets named fields
Point(x=100, y=22)
from collections import namedtuple 用法: namedtuple('名称', [属性list]): Circle = namedtuple('Circle', ['x', 'y', 'r']) 命名元组(namedtuple)有两个必需的参数。 它们是元组名称和字段名称。 namedtuple的每个实例没有对象字典,所以它们很轻量,与普通的元组相比,并不需要更多的内存。这使得它们比字典更快。属性值在namedtuple中是不可变的, namedtuple向后兼容于普通的元组,可以既使用整数索引,也可以使用名称来访问namedtuple ._asdict()方法将一个命名元组转换为字典,用法:namedtuple._asdict()
5.5 OrderedDict :字典的子类,保存了他们被添加的顺序 文档:
Help on class OrderedDict in module collections:
class OrderedDict(builtins.dict)
| Dictionary that remembers insertion order
|
| Method resolution order:
| OrderedDict
| builtins.dict
| builtins.object
|
| Methods defined here:
|
| __delitem__(self, key, /)
| Delete self[key].
|
| __eq__(self, value, /)
| Return self==value.
|
| __ge__(self, value, /)
| Return self>=value.
|
| __gt__(self, value, /)
| Return self>value.
|
| __init__(self, /, *args, **kwargs)
| Initialize self. See help(type(self)) for accurate signature.
|
| __iter__(self, /)
| Implement iter(self).
|
| __le__(self, value, /)
| Return self<=value.
|
| __lt__(self, value, /)
| Return self<value.
|
| __ne__(self, value, /)
| Return self!=value.
|
| __reduce__(...)
| Return state information for pickling
|
| __repr__(self, /)
| Return repr(self).
|
| __reversed__(...)
| od.__reversed__() <==> reversed(od)
|
| __setitem__(self, key, value, /)
| Set self[key] to value.
|
| __sizeof__(...)
| D.__sizeof__() -> size of D in memory, in bytes
|
| clear(...)
| od.clear() -> None. Remove all items from od.
|
| copy(...)
| od.copy() -> a shallow copy of od
|
| items(...)
| D.items() -> a set-like object providing a view on D's items
|
| keys(...)
| D.keys() -> a set-like object providing a view on D's keys
|
| move_to_end(self, /, key, last=True)
| Move an existing element to the end (or beginning if last is false).
|
| Raise KeyError if the element does not exist.
|
| pop(...)
| od.pop(k[,d]) -> v, remove specified key and return the corresponding
| value. If key is not found, d is returned if given, otherwise KeyError
| is raised.
|
| popitem(self, /, last=True)
| Remove and return a (key, value) pair from the dictionary.
|
| Pairs are returned in LIFO order if last is true or FIFO order if false.
|
| setdefault(self, /, key, default=None)
| Insert key with a value of default if key is not in the dictionary.
|
| Return the value for key if key is in the dictionary, else default.
|
| update(...)
| D.update([E, ]**F) -> None. Update D from dict/iterable E and F.
| If E is present and has a .keys() method, then does: for k in E: D[k] = E[k]
| If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v
| In either case, this is followed by: for k in F: D[k] = F[k]
|
| values(...)
| D.values() -> an object providing a view on D's values
|
| ----------------------------------------------------------------------
| Class methods defined here:
|
| fromkeys(iterable, value=None) from builtins.type
| Create a new ordered dictionary with keys from iterable and values set to value.
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| __hash__ = None
|
| ----------------------------------------------------------------------
| Methods inherited from builtins.dict:
|
| __contains__(self, key, /)
| True if the dictionary has the specified key, else False.
|
| __getattribute__(self, name, /)
| Return getattr(self, name).
|
| __getitem__(...)
| x.__getitem__(y) <==> x[y]
|
| __len__(self, /)
| Return len(self).
|
| get(self, key, default=None, /)
| Return the value for key if key is in the dictionary, else default.
|
| ----------------------------------------------------------------------
| Static methods inherited from builtins.dict:
|
| __new__(*args, **kwargs) from builtins.type
| Create and return a new object. See help(type) for accurate signature.
使用dict时,Key是无序的。在对dict做迭代时,我们无法确定Key的顺序。
如果要保持Key的顺序,可以用OrderedDict:
from collections import OrderedDict
OrderedDict 内部维护着一个根据键插入顺序排序的双向链表。每次当一个新的元素插入进来的时候,它会被放到链表的尾部。对于一个已经存在的键的重复赋值不会改变键的顺序。
OrderedDict的Key会按照插入的顺序排列,不是Key本身排序,即OrderedDict可以实现一个FIFO(先进先出)的dict,当容量超出限制时,先删除最早添加的Key。
需要注意的是,一个 OrderedDict 的大小是一个普通字典的两倍,因为它内部维护着另外一个链表。 所以如果你要构建一个需要大量OrderedDict实例的数据结构的时候(比如读取 100,000 行 CSV 数据到一个 OrderedDict 列表中去), 那么你就得仔细权衡一下是否使用 OrderedDict 带来的好处要大过额外内存消耗的影响。
【参考:1.7 字典排序 — python3-cookbook 3.0.0 文档 https://python3-cookbook.readthedocs.io python标准库之collections介绍 - luminousjj - 博客园 https://www.cnblogs.com/luminousjj/p/9342161.html】
6.Python itertools库:Python的内建模块itertools提供了非常有用的用于操作迭代对象的函数。
[参考文章:Python标准库之itertools库的使用方法_python_脚本之家 https://www.jb51.net/article/123094.htm] itertools中的函数大多是返回各种迭代器对象,其中很多函数的作用我们平时要写很多代码才能达到,而在运行效率上反而更低,毕竟人家是系统库。
itertools.accumulate 简单来说就是累加。
>>> import itertools
>>> x = itertools.accumulate(range(10))
>>> print(list(x))
[0, 1, 3, 6, 10, 15, 21, 28, 36, 45]
itertools.chain 连接多个列表或者迭代器。
>>> x = itertools.chain(range(3), range(4), [3,2,1])
>>> print(list(x))
[0, 1, 2, 0, 1, 2, 3, 3, 2, 1]
itertools.combinations 求列表或生成器中指定数目的元素不重复的所有组合
>>> x = itertools.combinations(range(4), 3)
>>> print(list(x))
[(0, 1, 2), (0, 1, 3), (0, 2, 3), (1, 2, 3)]
itertools.combinations_with_replacement 允许重复元素的组合
>>> x = itertools.combinations_with_replacement('ABC', 2)
>>> print(list(x))
[('A', 'A'), ('A', 'B'), ('A', 'C'), ('B', 'B'), ('B', 'C'), ('C', 'C')]
itertools.compress 按照真值表筛选元素
>>> x = itertools.compress(range(5), (True, False, True, True, False))
>>> print(list(x))
[0, 2, 3]
itertools.count 就是一个计数器,可以指定起始位置和步长
>>> x = itertools.count(start=20, step=-1)
>>> print(list(itertools.islice(x, 0, 10, 1)))
[20, 19, 18, 17, 16, 15, 14, 13, 12, 11]
itertools.cycle 循环指定的列表和迭代器
>>> x = itertools.cycle('ABC')
>>> print(list(itertools.islice(x, 0, 10, 1)))
['A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A']
itertools.dropwhile 按照真值函数丢弃掉列表和迭代器前面的元素
>>> x = itertools.dropwhile(lambda e: e < 5, range(10))
>>> print(list(x))
[5, 6, 7, 8, 9]
itertools.filterfalse 保留对应真值为False的元素
>>> x = itertools.filterfalse(lambda e: e < 5, (1, 5, 3, 6, 9, 4))
>>> print(list(x))
[5, 6, 9]
itertools.groupby 按照分组函数的值对元素进行分组
>>> x = itertools.groupby(range(10), lambda x: x < 5 or x > 8)
>>> for condition, numbers in x:
... print(condition, list(numbers))
True [0, 1, 2, 3, 4]
False [5, 6, 7, 8]
True [9]
itertools.islice 对迭代器进行切片
>>> x = itertools.islice(range(10), 0, 9, 2)
>>> print(list(x))
[0, 2, 4, 6, 8]
itertools.permutations 产生指定数目的元素的所有排列(顺序有关)
>>> x = itertools.permutations(range(4), 3)
>>> print(list(x))
[(0, 1, 2), (0, 1, 3), (0, 2, 1), (0, 2, 3), (0, 3, 1), (0, 3, 2), (1, 0, 2), (1, 0, 3), (1, 2, 0), (1, 2, 3), (1, 3, 0), (1, 3, 2), (2, 0, 1), (2, 0, 3), (2, 1, 0), (2, 1, 3), (2, 3, 0), (2, 3, 1), (3, 0, 1), (3, 0, 2), (3, 1, 0), (3, 1, 2), (3, 2, 0), (3, 2, 1)]
itertools.product 产生多个列表和迭代器的(积)
>>> x = itertools.product('ABC', range(3))
>>>
>>> print(list(x))
[('A', 0), ('A', 1), ('A', 2), ('B', 0), ('B', 1), ('B', 2), ('C', 0), ('C', 1), ('C', 2)]
itertools.repeat 简单的生成一个拥有指定数目元素的迭代器
>>> x = itertools.repeat(0, 5)
>>> print(list(x))
[0, 0, 0, 0, 0]
itertools.starmap 类似map
>>> x = itertools.starmap(str.islower, 'aBCDefGhI')
>>> print(list(x))
[True, False, False, False, True, True, False, True, False]
itertools.takewhile 与dropwhile相反,保留元素直至真值函数值为假。
>>> x = itertools.takewhile(lambda e: e < 5, range(10))
>>> print(list(x))
[0, 1, 2, 3, 4]
itertools.zip_longest 类似于zip,不过已较长的列表和迭代器的长度为准
>>> x = itertools.zip_longest(range(3), range(5))
>>> y = zip(range(3), range(5))
>>> print(list(x))
[(0, 0), (1, 1), (2, 2), (None, 3), (None, 4)]
>>> print(list(y))
[(0, 0), (1, 1), (2, 2)]
7.推导式 Comprehension ,可迭代对象 Iterable,迭代器 Iterator 及 生成器 Generators
7.1 推导式
推导式一般用的最多的是列表推导式(list comprehensions)
example:a = [i*i for i in range(10)]
官方解释:列表解析式是Python内置的非常简单却强大的可以用来创建list的生成式,在一个中括号里包含一个表达式, 然后是一个for语句, 然后是0个或多个for或者if语句。 那个表达式可以是任意的, 意思是你可以在列表中放任意类型的对象。返回结果将是一个新的列表,在这个以if和for语句为上下文的表达式运行完成之后产生。
可以看到,使用列表解析式的写法更加简短,除此之外,因为是Python内置的用法,底层使用c语言实现,相较于编写Python代码而言,运行速度更快。
====总结====:如果要对现有可迭代对象做一下处理,然后生成新的列表,使用列表推导式将是最便捷的方法。
7.2 可迭代对象 Iterable
最简单的解释:可以使用for...in...语句进行循环的对象,就是可迭代对象(Iterable),可以使用isinstance()方法进行判断。list、dict、str是Iterable(可迭代对象),但不是Iterator(迭代器),可以使用iter()函数把list、dict、str等Iterable变成Iterator。
temp=iter([1, 2, 3]) print type(temp) # list_iterator print next(temp) # 1
7.3 迭代器 Iterator
任意对象,只要定义了next (Python2) 或者next方法并实现了iter方法,它就是一个迭代器。 迭代器的优势:在构建迭代器时,不是将所有元素一次性的加载,而是等调用next方法时返回元素,大量节省内存。 迭代器应用场景:数列的数据规模巨大且数列有规律,但是不能使用列表推导式描述的时候,如大数据量的数列的创建,大文件的读取等。
7.4 生成器 Generators
生成器是一个特殊的函数,可以被用作控制循环的迭代行为,python中生成器是一种高级迭代器,使用yield代替return返回,每次调用yield会暂停,而可以使用next()函数和send()函数恢复生成器。 生成器类似于返回值为一个数组的一个函数,这个函数可以接收参数,可以被调用,但是不同于一般的函数会一次性返回包括了所有数值的数组,生成器一次只能产生一个值,这样消耗的内存数量将大大减小,生成器可以被看作是一个函数,但是它本质上也算是一个迭代器。
生成器函数:为什么叫生成器函数?因为它随着时间的推移生成了一个数值队列。一般的函数在执行完毕之后会返回一个值然后退出,但是生成器函数会自动挂起,然后重新拾起急需执行,利用yield关键字关起函数,给调用者返回一个值,同时保留了当前的足够多的状态,可以使函数继续执行。当需要一个将返回一个序列或在循环中执行的函数时,就可以使用生成器,因为当这些元素被传递到另一个函数中进行后续处理时,一次返回一个元素可以有效的提升整体性能。常见的应用场景是使用生成器生成数据流缓冲区。
生成器表达式:生成式表达式是一种实现生成器的便捷方式,将列表推导式的中括号替换为圆括号。返回一个对象,这个对象只有在需要的时候才产生结果。 生成器表达式的一个例子: g = (xx for x in range(10))
[参考文章]: python 生成器和迭代器有这篇就够了 - 战争热诚 - 博客园 https://www.cnblogs.com/wj-1314/p/8490822.html 迭代器 生成器, 可迭代对象以及应用场景 - 若无过客丶何来人生 - 博客园 https://www.cnblogs.com/liurenli/p/10127818.html
8.命名切片
slice() 函数 描述:slice() 函数用实现切片对象,主要在切片操作函数里的参数传递。 slice 语法: class slice(stop) class slice(start, stop[, step])
如果你有一个切片对象 a,你可以分别调用它的 a.start , a.stop , a.step 属性来获取更多的信息。 另外,你还能通过调用切片的 indices(size) 方法将它映射到一个确定大小的序列上,这个方法返回一个三元组 (start, stop, step) ,所有值都会被合适的缩小以满足边界限制,从而使用的时候避免出现 IndexError 异常。
文档的说明: class slice(object) | slice(stop) | slice(start, stop[, step]) | | Create a slice object. This is used for extended slicing (e.g. a[0:10:2]). | | Methods defined here: | | eq(self, value, /) | Return self==value. | | ge(self, value, /) | Return self>=value. | | getattribute(self, name, /) | Return getattr(self, name). | | gt(self, value, /) | Return self>value. | | le(self, value, /) | Return self<=value. | | lt(self, value, /) | Return self<value. | | ne(self, value, /) | Return self!=value. | | reduce(...) | Return state information for pickling. | | repr(self, /) | Return repr(self). | | indices(...) | S.indices(len) -> (start, stop, stride) | | Assuming a sequence of length len, calculate the start and stop | indices, and the stride length of the extended slice described by | S. Out of bounds indices are clipped in a manner consistent with the | handling of normal slices. | | ---------------------------------------------------------------------- | Static methods defined here: | | new(*args, **kwargs) from builtins.type | Create and return a new object. See help(type) for accurate signature.-- More -- | | ---------------------------------------------------------------------- | Data descriptors defined here: | | start | | step | | stop | | ---------------------------------------------------------------------- | Data and other attributes defined here: | | hash = None
9.上下文管理器:
常见用法: with open('test.txt') as f: print f.readlines()
几个概念
- 上下文表达式:with open('test.txt') as f:
- 上下文管理器:open('test.txt')
- f 不是上下文管理器,应该是资源对象。 简单点说,就是在一个类里,实现了enter和exit的方法,这个类的实例就是一个上下文管理器。
为什么要使用上下文管理器?
- 可以以一种更加优雅的方式,操作(创建/获取/释放)资源,如文件操作、数据库连接;
- 可以以一种更加优雅的方式,处理异常;
一个基于类实现的上下文理器: class File(object): def init(self, file_name, method): self.file_obj = open(file_name, method) def enter(self): return self.file_obj def exit(self, exc_type, exc_value, exc_traceback): self.file_obj.close()
注:exc_type:异常类型 , exc_value:异常值, exc_traceback:异常的错误栈信息
执行过程:
with语句先暂存了File类的exit方法; 然后它调用File类的enter方法; enter方法打开文件并返回给with语句; 打开的文件句柄被传递给opened_file参数; 我们使用.write()来写文件; with语句调用之前暂存的exit方法; exit方法关闭了文件。
[关于上下文管理器的更多详细可以参考文章]: 浅淡python中with的用法,上下文管理器 - 听风。 - 博客园 https://www.cnblogs.com/huchong/p/8268765.html 深入理解 Python 中的上下文管理器 - 王一白 - 博客园 https://www.cnblogs.com/wongbingming/p/10519553.html
统一声明:关于原创博客内容,可能会有部分内容参考自互联网,如有原创链接会声明引用;如找不到原创链接,在此声明如有侵权请联系删除哈。关于转载博客,如有原创链接会声明;如找不到原创链接,如果有哪些问题、有些谬误、有哪些不妥或者侵犯到您的权益的地方,可以联系我,我马上修改。