Python builtin 函数及重要知识点学习笔记(一)


《Python进阶》学习笔记
1. *args 和 **kwargs

*args 是用来发送一定长度的键值对, 作为参数传递给一个函数。 **kwargs 允许你将不定长度的键值对, 作为参数传递给⼀个函数。

2. 调试 Debugging

命令行使用Python debugger运行一个脚本: $ python -m pdb my_script.py pdb模块(断点调试)用法:   import pdb   功能:断点设置,单步执行,查看代码,查看函数,追踪变量等   pdb.set_trace()   功能 : 设置初始断点,开始进入pdb调试模式调试中的命令   break , b : 设置断点   continue ,c : 继续执行 w , : 显示当前正在执行的代码行的上下文信息   list , l : 查看当前代码段   next, n : 单步执行   step, s : 进入函数单步执行   pp : 打印变量值   help : 帮助   q : 退出调试   exit : 退出pdb调试

3.set 数据结构

交集:你可以对比两个集合的交集(两个集合中都有的数据) set_a.intersection(set_b) : set_a和set_b共同拥有的元素,以set形式返回; 差集:你可以用差集(difference)找出无效的数据,相当于用一个集合减去另一个集合的数据 set_a.difference(set_b) : set_a减去set_b,相当于set_a单独拥有的元素,以set形式返回; 也可以用{}符号来创建集合,如: a_set = {'red', 'blue', 'green'} print(type(a_set)) 输出: <type 'set'>

4. slots魔法方法

slots来告诉Python,当一个类需要创建大量实例时,可以通过slots声明实例所需要的属性,不要使用字典,而且只给一个固定集合的属性分配空间。 这样做带来以下优点:

  • 更快的属性访问速度
  • 减少内存消耗
  • 利用这种特性来限制实例的属性。

默认情况下,访问一个实例的属性是通过访问该实例的dict来实现的。如访问a.x就相当于访问a.dict['x']。为了便于理解,我粗略地将它拆分为四步: a.x 2. a.dict 3. a.dict['x'] 4. 结果

slots的实现可以得知,定义了slots的类会为每个属性创建一个描述器。访问属性时就直接调用这个描述器。在这里我将它拆分为三步: b.x 2. member decriptor 3. 结果

我在上文提到,访问dict和描述器的速度是相近的,而通过dict访问属性多了a.dict['x']字典访值一步(一个哈希函数的消耗)。由此可以推断出,使用了slots的类的属性访问速度比没有使用的要快。 [参考文章:Pythonslots详解 - rainfd - 博客园 https://www.cnblogs.com/rainfd/p/slots.html]

5.容器数据类型 Collections

重点: 5.1 defaultdict :字典的子类,提供了一个工厂函数,为字典查询提供一个默认值; 文档:

Help on class defaultdict in module collections:
class defaultdict(builtins.dict)
 |  defaultdict(default_factory[, ...]) --> dict with default factory
 |
 |  The default factory is called without arguments to produce
 |  a new value when a key is not present, in __getitem__ only.
 |  A defaultdict compares equal to a dict with the same items.
 |  All remaining arguments are treated the same as if they were
 |  passed to the dict constructor, including keyword arguments.
 |
 |  Method resolution order:
 |      defaultdict
 |      builtins.dict
 |      builtins.object
 |
 |  Methods defined here:
 |
 |  __copy__(...)
 |      D.copy() -> a shallow copy of D.
 |
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |
 |  __init__(self, /, *args, **kwargs)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |
 |  __missing__(...)
 |      __missing__(key) # Called by __getitem__ for missing key; pseudo-code:
 |      if self.default_factory is None: raise KeyError((key,))
 |      self[key] = value = self.default_factory()
 |      return value
 |
 |  __reduce__(...)
 |      Return state information for pickling.
 |
 |  __repr__(self, /)
 |      Return repr(self).
 |
 |  copy(...)
 |      D.copy() -> a shallow copy of D.
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  default_factory
 |      Factory for default value called by __missing__().
 |
 |  ----------------------------------------------------------------------
 |  Methods inherited from builtins.dict:
 |
 |  __contains__(self, key, /)
 |      True if the dictionary has the specified key, else False.
 |
 |  __delitem__(self, key, /)
 |      Delete self[key].
 |
 |  __eq__(self, value, /)
 |      Return self==value.
 |
 |  __ge__(self, value, /)
 |      Return self>=value.
 |
 |  __getitem__(...)
 |      x.__getitem__(y) <==> x[y]
 |
 |  __gt__(self, value, /)
 |      Return self>value.
 |
 |  __iter__(self, /)
 |      Implement iter(self).
 |
 |  __le__(self, value, /)
 |      Return self<=value.
 |
 |  __len__(self, /)
 |      Return len(self).
 |
 |  __lt__(self, value, /)
 |      Return self<value.
 |
 |  __ne__(self, value, /)
 |      Return self!=value.
 |
 |  __setitem__(self, key, value, /)
 |      Set self[key] to value.
 |
 |  __sizeof__(...)
 |      D.__sizeof__() -> size of D in memory, in bytes
 |
 |  clear(...)
 |      D.clear() -> None.  Remove all items from D.
 |
 |  get(self, key, default=None, /)
 |      Return the value for key if key is in the dictionary, else default.
 |
 |  items(...)
 |      D.items() -> a set-like object providing a view on D's items
 |
 |  keys(...)
 |      D.keys() -> a set-like object providing a view on D's keys
 |
 |  pop(...)
 |      D.pop(k[,d]) -> v, remove specified key and return the corresponding value.
 |      If key is not found, d is returned if given, otherwise KeyError is raised
 |
 |  popitem(...)
 |      D.popitem() -> (k, v), remove and return some (key, value) pair as a
 |      2-tuple; but raise KeyError if D is empty.
 |
 |  setdefault(self, key, default=None, /)
 |      Insert key with a value of default if key is not in the dictionary.
 |
 |      Return the value for key if key is in the dictionary, else default.
 |
 |  update(...)
 |      D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.
 |      If E is present and has a .keys() method, then does:  for k in E: D[k] = E[k]
 |      If E is present and lacks a .keys() method, then does:  for k, v in E: D[k] = v
 |      In either case, this is followed by: for k in F:  D[k] = F[k]
 |
 |  values(...)
 |      D.values() -> an object providing a view on D's values
 |
 |  ----------------------------------------------------------------------
 |  Class methods inherited from builtins.dict:
 |
 |  fromkeys(iterable, value=None, /) from builtins.type
 |      Create a new dictionary with keys from iterable and values set to value.
 |
 |  ----------------------------------------------------------------------
 |  Static methods inherited from builtins.dict:
 |
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.
 |
 |  ----------------------------------------------------------------------
 |  Data and other attributes inherited from builtins.dict:
 |
 |  __hash__ = None

与一般dict区别:标准字典包括setdefault方法()获取一个值,如果值不存在,建立一个默认; 相比之下,defaultdict允许调用者在初始化时预先设置默认值。

>>> from collections import defaultdict
>>> dd = defaultdict(lambda: 'N/A')
>>> dd['key1'] = 'abc'
>>> dd['key1'] # key1存在
'abc'
>>> dd['key2'] # key2不存在,返回默认值
'N/A'

使用dict时,如果引用的Key不存在,就会抛出KeyError。如果希望key不存在时,返回一个默认值,就可以用defaultdict 注意:默认值是调用函数返回的,而函数在创建defaultdict对象时传入。

5.2 Counter:字典的子类,提供了可哈希对象的计数功能

文档:

class Counter(builtins.dict)
 |  Counter(*args, **kwds)
 |
 |  Dict subclass for counting hashable items.  Sometimes called a bag
 |  or multiset.  Elements are stored as dictionary keys and their counts
 |  are stored as dictionary values.
 |
 |  >>> c = Counter('abcdeabcdabcaba')  # count elements from a string
 |
 |  >>> c.most_common(3)                # three most common elements
 |  [('a', 5), ('b', 4), ('c', 3)]
 |  >>> sorted(c)                       # list all unique elements
 |  ['a', 'b', 'c', 'd', 'e']
 |  >>> ''.join(sorted(c.elements()))   # list elements with repetitions
 |  'aaaaabbbbcccdde'
 |  >>> sum(c.values())                 # total of all counts
 |  15
 |
 |  >>> c['a']                          # count of letter 'a'
 |  5
 |  >>> for elem in 'shazam':           # update counts from an iterable
 |  ...     c[elem] += 1                # by adding 1 to each element's count
 |  >>> c['a']                          # now there are seven 'a'
 |  7
 |  >>> del c['b']                      # remove all 'b'
 |  >>> c['b']                          # now there are zero 'b'
 |  0
 |
 |  >>> d = Counter('simsalabim')       # make another counter
 |  >>> c.update(d)                     # add in the second counter
 |  >>> c['a']                          # now there are nine 'a'
 |  9
 |
 |  >>> c.clear()                       # empty the counter
 |  >>> c
 |  Counter()
 |
 |  Note:  If a count is set to zero or reduced to zero, it will remain
 |  in the counter until the entry is deleted or the counter is cleared:
 |
 |  >>> c = Counter('aaabbc')
 |  >>> c['b'] -= 2                     # reduce the count of 'b' by two
 |  >>> c.most_common()                 # 'b' is still in, but its count is zero
 |  [('a', 3), ('c', 1), ('b', 0)]
 |
 |  Method resolution order:
 |      Counter
 |      builtins.dict
 |      builtins.object
 |
 |  Methods defined here:
 |
 |  __add__(self, other)
 |      Add counts from two counters.
 |
 |      >>> Counter('abbb') + Counter('bcc')
 |      Counter({'b': 4, 'c': 2, 'a': 1})
 |
 |  __and__(self, other)
 |      Intersection is the minimum of corresponding counts.
 |
 |      >>> Counter('abbb') & Counter('bcc')
 |      Counter({'b': 1})
 |
 |  __delitem__(self, elem)
 |      Like dict.__delitem__() but does not raise KeyError for missing values.
 |
 |  __iadd__(self, other)
 |      Inplace add from another counter, keeping only positive counts.
 |
 |      >>> c = Counter('abbb')
 |      >>> c += Counter('bcc')
 |      >>> c
 |      Counter({'b': 4, 'c': 2, 'a': 1})
 |
 |  __iand__(self, other)
 |      Inplace intersection is the minimum of corresponding counts.
 |
 |      >>> c = Counter('abbb')
 |      >>> c &= Counter('bcc')
 |      >>> c
 |      Counter({'b': 1})
 |
 |  __init__(*args, **kwds)
 |      Create a new, empty Counter object.  And if given, count elements
 |      from an input iterable.  Or, initialize the count from another mapping
 |      of elements to their counts.
 |
 |      >>> c = Counter()                           # a new, empty counter
 |      >>> c = Counter('gallahad')                 # a new counter from an iterable
 |      >>> c = Counter({'a': 4, 'b': 2})           # a new counter from a mapping
 |      >>> c = Counter(a=4, b=2)                   # a new counter from keyword args
 |
 |  __ior__(self, other)
 |      Inplace union is the maximum of value from either counter.
 |
 |      >>> c = Counter('abbb')
 |      >>> c |= Counter('bcc')
 |      >>> c
 |      Counter({'b': 3, 'c': 2, 'a': 1})
 |
 |  __isub__(self, other)
 |      Inplace subtract counter, but keep only results with positive counts.
 |
 |      >>> c = Counter('abbbc')
 |      >>> c -= Counter('bccd')
 |      >>> c
 |      Counter({'b': 2, 'a': 1})
 |
 |  __missing__(self, key)
 |      The count of elements not in the Counter is zero.
 |
 |  __neg__(self)
 |      Subtracts from an empty counter.  Strips positive and zero counts,
 |      and flips the sign on negative counts.
 |
 |  __or__(self, other)
 |      Union is the maximum of value in either of the input counters.
 |
 |      >>> Counter('abbb') | Counter('bcc')
 |      Counter({'b': 3, 'c': 2, 'a': 1})
 |
 |  __pos__(self)
 |      Adds an empty counter, effectively stripping negative and zero counts
 |
 |  __reduce__(self)
 |      Helper for pickle.
 |
 |  __repr__(self)
 |      Return repr(self).
 |
 |  __sub__(self, other)
 |      Subtract count, but keep only results with positive counts.
 |
 |      >>> Counter('abbbc') - Counter('bccd')
 |      Counter({'b': 2, 'a': 1})
 |
 |  copy(self)
 |      Return a shallow copy.
 |
 |  elements(self)
 |      Iterator over elements repeating each as many times as its count.
 |
 |      >>> c = Counter('ABCABC')
 |      >>> sorted(c.elements())
 |      ['A', 'A', 'B', 'B', 'C', 'C']
 |
 |      # Knuth's example for prime factors of 1836:  2**2 * 3**3 * 17**1
 |      >>> prime_factors = Counter({2: 2, 3: 3, 17: 1})
 |      >>> product = 1
 |      >>> for factor in prime_factors.elements():     # loop over factors
 |      ...     product *= factor                       # and multiply them
 |      >>> product
 |      1836
 |
 |      Note, if an element's count has been set to zero or is a negative
 |      number, elements() will ignore it.
 |
 |  most_common(self, n=None)
 |      List the n most common elements and their counts from the most
 |      common to the least.  If n is None, then list all element counts.
 |
 |      >>> Counter('abcdeabcdabcaba').most_common(3)
 |      [('a', 5), ('b', 4), ('c', 3)]
 |
 |  subtract(*args, **kwds)
 |      Like dict.update() but subtracts counts instead of replacing them.
 |      Counts can be reduced below zero.  Both the inputs and outputs are
 |      allowed to contain zero and negative counts.
 |
 |      Source can be an iterable, a dictionary, or another Counter instance.
 |
 |      >>> c = Counter('which')
 |      >>> c.subtract('witch')             # subtract elements from another iterable
 |      >>> c.subtract(Counter('watch'))    # subtract elements from another counter
 |      >>> c['h']                          # 2 in which, minus 1 in witch, minus 1 in watch
 |      0
 |      >>> c['w']                          # 1 in which, minus 1 in witch, minus 1 in watch
 |      -1
 |
 |  update(*args, **kwds)
 |      Like dict.update() but add counts instead of replacing them.
 |
 |      Source can be an iterable, a dictionary, or another Counter instance.
 |
 |      >>> c = Counter('which')
 |      >>> c.update('witch')           # add elements from another iterable
 |      >>> d = Counter('watch')
 |      >>> c.update(d)                 # add elements from another counter
 |      >>> c['h']                      # four 'h' in which, witch, and watch
 |      4
 |
 |  ----------------------------------------------------------------------
 |  Class methods defined here:
 |
 |  fromkeys(iterable, v=None) from builtins.type
 |      Create a new dictionary with keys from iterable and values set to value.
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  __dict__
 |      dictionary for instance variables (if defined)
 |
 |  __weakref__
 |      list of weak references to the object (if defined)
 |
 |  ----------------------------------------------------------------------
 |  Methods inherited from builtins.dict:
 |
 |  __contains__(self, key, /)
 |      True if the dictionary has the specified key, else False.
 |
 |  __eq__(self, value, /)
 |      Return self==value.
 |
 |  __ge__(self, value, /)
 |      Return self>=value.
 |
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |
 |  __getitem__(...)
 |      x.__getitem__(y) <==> x[y]
 |
 |  __gt__(self, value, /)
 |      Return self>value.
 |
 |  __iter__(self, /)
 |      Implement iter(self).
 |
 |  __le__(self, value, /)
 |      Return self<=value.
 |
 |  __len__(self, /)
 |      Return len(self).
 |
 |  __lt__(self, value, /)
 |      Return self<value.
 |
 |  __ne__(self, value, /)
 |      Return self!=value.
 |
 |  __setitem__(self, key, value, /)
 |      Set self[key] to value.
 |
 |  __sizeof__(...)
 |      D.__sizeof__() -> size of D in memory, in bytes
 |
 |  clear(...)
 |      D.clear() -> None.  Remove all items from D.
 |
 |  get(self, key, default=None, /)
 |      Return the value for key if key is in the dictionary, else default.
 |
 |  items(...)
 |      D.items() -> a set-like object providing a view on D's items
 |
 |  keys(...)
 |      D.keys() -> a set-like object providing a view on D's keys
 |
 |  pop(...)
 |      D.pop(k[,d]) -> v, remove specified key and return the corresponding value.
 |      If key is not found, d is returned if given, otherwise KeyError is raised
 |
 |  popitem(...)
 |      D.popitem() -> (k, v), remove and return some (key, value) pair as a
 |      2-tuple; but raise KeyError if D is empty.
 |
 |  setdefault(self, key, default=None, /)
 |      Insert key with a value of default if key is not in the dictionary.
 |
 |      Return the value for key if key is in the dictionary, else default.
 |
 |  values(...)
 |      D.values() -> an object providing a view on D's values
 |
 |  ----------------------------------------------------------------------
 |  Static methods inherited from builtins.dict:
 |
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.
 |
 |  ----------------------------------------------------------------------
 |  Data and other attributes inherited from builtins.dict:
 |
 |  __hash__ = None

from collections import Counter

Counter是一个简单的计数器,例如,统计字符出现的个数,空的Counter容器可以无参数构造,并采用update()方法进行更新;

most_common()返回前n个最多的数据,counter.most_common(x)返回前x个最多的数据;

Counter实例支持聚合结果的算术和集合操作;

elements()方法可以返回一个包含所有Counter数据的迭代器,counter.elements()。

5.3 deque 双端队列 :

文档:

class deque(builtins.object)
 |  deque([iterable[, maxlen]]) --> deque object
 |
 |  A list-like sequence optimized for data accesses near its endpoints.
 |
 |  Methods defined here:
 |
 |  __add__(self, value, /)
 |      Return self+value.
 |
 |  __bool__(self, /)
 |      self != 0
 |
 |  __contains__(self, key, /)
 |      Return key in self.
 |
 |  __copy__(...)
 |      Return a shallow copy of a deque.
 |
 |  __delitem__(self, key, /)
 |      Delete self[key].
 |
 |  __eq__(self, value, /)
 |      Return self==value.
 |
 |  __ge__(self, value, /)
 |      Return self>=value.
 |
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |
 |  __getitem__(self, key, /)
 |      Return self[key].
 |
 |  __gt__(self, value, /)
 |      Return self>value.
 |
 |  __iadd__(self, value, /)
 |      Implement self+=value.
 |
 |  __imul__(self, value, /)
 |      Implement self*=value.
 |
 |  __init__(self, /, *args, **kwargs)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |
 |  __iter__(self, /)
 |      Implement iter(self).
 |
 |  __le__(self, value, /)
 |      Return self<=value.
 |
 |  __len__(self, /)
 |      Return len(self).
 |
 |  __lt__(self, value, /)
 |      Return self<value.
 |
 |  __mul__(self, value, /)
 |      Return self*value.
 |
 |  __ne__(self, value, /)
 |      Return self!=value.
 |
 |  __reduce__(...)
 |      Return state information for pickling.
 |
 |  __repr__(self, /)
 |      Return repr(self).
 |
 |  __reversed__(...)
 |      D.__reversed__() -- return a reverse iterator over the deque
 |
 |  __rmul__(self, value, /)
 |      Return value*self.
 |
 |  __setitem__(self, key, value, /)
 |      Set self[key] to value.
 |
 |  __sizeof__(...)
 |      D.__sizeof__() -- size of D in memory, in bytes
 |
 |  append(...)
 |      Add an element to the right side of the deque.
 |
 |  appendleft(...)
 |      Add an element to the left side of the deque.
 |
 |  clear(...)
 |      Remove all elements from the deque.
 |
 |  copy(...)
 |      Return a shallow copy of a deque.
 |
 |  count(...)
 |      D.count(value) -> integer -- return number of occurrences of value
 |
 |  extend(...)
 |      Extend the right side of the deque with elements from the iterable
 |
 |  extendleft(...)
 |      Extend the left side of the deque with elements from the iterable
 |
 |  index(...)
 |      D.index(value, [start, [stop]]) -> integer -- return first index of value.
 |      Raises ValueError if the value is not present.
 |
 |  insert(...)
 |      D.insert(index, object) -- insert object before index
 |
 |  pop(...)
 |      Remove and return the rightmost element.
 |
 |  popleft(...)
 |      Remove and return the leftmost element.
 |
 |  remove(...)
 |      D.remove(value) -- remove first occurrence of value.
 |
 |  reverse(...)
 |      D.reverse() -- reverse *IN PLACE*
 |
 |  rotate(...)
 |      Rotate the deque n steps to the right (default n=1).  If n is negative, rotates left.
 |
 |  ----------------------------------------------------------------------
 |  Static methods defined here:
 |
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  maxlen
 |      maximum size of a deque or None if unbounded
 |
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |
 |  __hash__ = None

类似列表(list)的容器,实现了在两端快速添加(append)和弹出(pop),deque是为了高效实现插入和删除操作的双向列表,适合用于队列和栈, deque除了实现list的append()和pop()外,还支持左端插入appendleft()和左端删除popleft(),这样就可以非常高效地往头部添加或 删除元素。 由于双端队列是线程安全的,在单独的线程中内容甚至可以从两端同时消费。

5.4 namedtuple 命名元组 :创建命名元组子类的工厂函数

文档:

Help on function namedtuple in module collections:
namedtuple(typename, field_names, *, rename=False, defaults=None, module=None)
    Returns a new subclass of tuple with named fields.

    >>> Point = namedtuple('Point', ['x', 'y'])
    >>> Point.__doc__                   # docstring for the new class
    'Point(x, y)'
    >>> p = Point(11, y=22)             # instantiate with positional args or keywords    >>> p[0] + p[1]                     # indexable 

like a plain tuple
    33
    >>> x, y = p                        # unpack like a regular tuple
    >>> x, y
    (11, 22)
    >>> p.x + p.y                       # fields also accessible by name
    33
    >>> d = p._asdict()                 # convert to a dictionary
    >>> d['x']
    11
    >>> Point(**d)                      # convert from a dictionary
    Point(x=11, y=22)
    >>> p._replace(x=100)               # _replace() is like str.replace() but targets named fields
    Point(x=100, y=22)

from collections import namedtuple 用法: namedtuple('名称', [属性list]): Circle = namedtuple('Circle', ['x', 'y', 'r']) 命名元组(namedtuple)有两个必需的参数。 它们是元组名称和字段名称。 namedtuple的每个实例没有对象字典,所以它们很轻量,与普通的元组相比,并不需要更多的内存。这使得它们比字典更快。属性值在namedtuple中是不可变的, namedtuple向后兼容于普通的元组,可以既使用整数索引,也可以使用名称来访问namedtuple ._asdict()方法将一个命名元组转换为字典,用法:namedtuple._asdict()

5.5 OrderedDict :字典的子类,保存了他们被添加的顺序 文档:

Help on class OrderedDict in module collections:

class OrderedDict(builtins.dict)
 |  Dictionary that remembers insertion order
 |
 |  Method resolution order:
 |      OrderedDict
 |      builtins.dict
 |      builtins.object
 |
 |  Methods defined here:
 |
 |  __delitem__(self, key, /)
 |      Delete self[key].
 |
 |  __eq__(self, value, /)
 |      Return self==value.
 |
 |  __ge__(self, value, /)
 |      Return self>=value.
 |
 |  __gt__(self, value, /)
 |      Return self>value.
 |
 |  __init__(self, /, *args, **kwargs)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |
 |  __iter__(self, /)
 |      Implement iter(self).
 |
 |  __le__(self, value, /)
 |      Return self<=value.
 |
 |  __lt__(self, value, /)
 |      Return self<value.
 |
 |  __ne__(self, value, /)
 |      Return self!=value.
 |
 |  __reduce__(...)
 |      Return state information for pickling
 |
 |  __repr__(self, /)
 |      Return repr(self).
 |
 |  __reversed__(...)
 |      od.__reversed__() <==> reversed(od)
 |
 |  __setitem__(self, key, value, /)
 |      Set self[key] to value.
 |
 |  __sizeof__(...)
 |      D.__sizeof__() -> size of D in memory, in bytes
 |
 |  clear(...)
 |      od.clear() -> None.  Remove all items from od.
 |
 |  copy(...)
 |      od.copy() -> a shallow copy of od
 |
 |  items(...)
 |      D.items() -> a set-like object providing a view on D's items
 |
 |  keys(...)
 |      D.keys() -> a set-like object providing a view on D's keys
 |
 |  move_to_end(self, /, key, last=True)
 |      Move an existing element to the end (or beginning if last is false).
 |
 |      Raise KeyError if the element does not exist.
 |
 |  pop(...)
 |      od.pop(k[,d]) -> v, remove specified key and return the corresponding
 |      value.  If key is not found, d is returned if given, otherwise KeyError
 |      is raised.
 |
 |  popitem(self, /, last=True)
 |      Remove and return a (key, value) pair from the dictionary.
 |
 |      Pairs are returned in LIFO order if last is true or FIFO order if false.
 |
 |  setdefault(self, /, key, default=None)
 |      Insert key with a value of default if key is not in the dictionary.
 |
 |      Return the value for key if key is in the dictionary, else default.
 |
 |  update(...)
 |      D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.
 |      If E is present and has a .keys() method, then does:  for k in E: D[k] = E[k]
 |      If E is present and lacks a .keys() method, then does:  for k, v in E: D[k] = v
 |      In either case, this is followed by: for k in F:  D[k] = F[k]
 |
 |  values(...)
 |      D.values() -> an object providing a view on D's values
 |
 |  ----------------------------------------------------------------------
 |  Class methods defined here:
 |
 |  fromkeys(iterable, value=None) from builtins.type
 |      Create a new ordered dictionary with keys from iterable and values set to value.
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  __dict__
 |
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |
 |  __hash__ = None
 |
 |  ----------------------------------------------------------------------
 |  Methods inherited from builtins.dict:
 |
 |  __contains__(self, key, /)
 |      True if the dictionary has the specified key, else False.
 |
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |
 |  __getitem__(...)
 |      x.__getitem__(y) <==> x[y]
 |
 |  __len__(self, /)
 |      Return len(self).
 |
 |  get(self, key, default=None, /)
 |      Return the value for key if key is in the dictionary, else default.
 |
 |  ----------------------------------------------------------------------
 |  Static methods inherited from builtins.dict:
 |
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.

使用dict时,Key是无序的。在对dict做迭代时,我们无法确定Key的顺序。

如果要保持Key的顺序,可以用OrderedDict:

from collections import OrderedDict

OrderedDict 内部维护着一个根据键插入顺序排序的双向链表。每次当一个新的元素插入进来的时候,它会被放到链表的尾部。对于一个已经存在的键的重复赋值不会改变键的顺序。

OrderedDict的Key会按照插入的顺序排列,不是Key本身排序,即OrderedDict可以实现一个FIFO(先进先出)的dict,当容量超出限制时,先删除最早添加的Key。

需要注意的是,一个 OrderedDict 的大小是一个普通字典的两倍,因为它内部维护着另外一个链表。 所以如果你要构建一个需要大量OrderedDict实例的数据结构的时候(比如读取 100,000 行 CSV 数据到一个 OrderedDict 列表中去), 那么你就得仔细权衡一下是否使用 OrderedDict 带来的好处要大过额外内存消耗的影响。

【参考:1.7 字典排序 — python3-cookbook 3.0.0 文档 https://python3-cookbook.readthedocs.io python标准库之collections介绍 - luminousjj - 博客园 https://www.cnblogs.com/luminousjj/p/9342161.html】

6.Python itertools库:Python的内建模块itertools提供了非常有用的用于操作迭代对象的函数。

[参考文章:Python标准库之itertools库的使用方法_python_脚本之家 https://www.jb51.net/article/123094.htm] itertools中的函数大多是返回各种迭代器对象,其中很多函数的作用我们平时要写很多代码才能达到,而在运行效率上反而更低,毕竟人家是系统库。

itertools.accumulate 简单来说就是累加。

>>> import itertools
>>> x = itertools.accumulate(range(10))
>>> print(list(x))
[0, 1, 3, 6, 10, 15, 21, 28, 36, 45]

itertools.chain 连接多个列表或者迭代器。

>>> x = itertools.chain(range(3), range(4), [3,2,1])
>>> print(list(x))
[0, 1, 2, 0, 1, 2, 3, 3, 2, 1]

itertools.combinations 求列表或生成器中指定数目的元素不重复的所有组合

>>> x = itertools.combinations(range(4), 3)
>>> print(list(x))
[(0, 1, 2), (0, 1, 3), (0, 2, 3), (1, 2, 3)]

itertools.combinations_with_replacement 允许重复元素的组合

>>> x = itertools.combinations_with_replacement('ABC', 2)
>>> print(list(x))
[('A', 'A'), ('A', 'B'), ('A', 'C'), ('B', 'B'), ('B', 'C'), ('C', 'C')]

itertools.compress 按照真值表筛选元素

>>> x = itertools.compress(range(5), (True, False, True, True, False))
>>> print(list(x))
[0, 2, 3]

itertools.count 就是一个计数器,可以指定起始位置和步长

>>> x = itertools.count(start=20, step=-1)
>>> print(list(itertools.islice(x, 0, 10, 1)))
[20, 19, 18, 17, 16, 15, 14, 13, 12, 11]

itertools.cycle 循环指定的列表和迭代器

>>> x = itertools.cycle('ABC')
>>> print(list(itertools.islice(x, 0, 10, 1)))
['A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A']

itertools.dropwhile 按照真值函数丢弃掉列表和迭代器前面的元素

>>> x = itertools.dropwhile(lambda e: e < 5, range(10))
>>> print(list(x))
[5, 6, 7, 8, 9]

itertools.filterfalse 保留对应真值为False的元素

>>> x = itertools.filterfalse(lambda e: e < 5, (1, 5, 3, 6, 9, 4))
>>> print(list(x))
[5, 6, 9]

itertools.groupby 按照分组函数的值对元素进行分组

>>> x = itertools.groupby(range(10), lambda x: x < 5 or x > 8)            
>>> for condition, numbers in x:      
... print(condition, list(numbers))            
True [0, 1, 2, 3, 4]       
False [5, 6, 7, 8]       
True [9]

itertools.islice 对迭代器进行切片

>>> x = itertools.islice(range(10), 0, 9, 2)
>>> print(list(x))
[0, 2, 4, 6, 8]

itertools.permutations 产生指定数目的元素的所有排列(顺序有关)

>>> x = itertools.permutations(range(4), 3)
>>> print(list(x))
[(0, 1, 2), (0, 1, 3), (0, 2, 1), (0, 2, 3), (0, 3, 1), (0, 3, 2), (1, 0, 2), (1, 0, 3), (1, 2, 0), (1, 2, 3), (1, 3, 0), (1, 3, 2), (2, 0, 1), (2, 0, 3), (2, 1, 0), (2, 1, 3), (2, 3, 0), (2, 3, 1), (3, 0, 1), (3, 0, 2), (3, 1, 0), (3, 1, 2), (3, 2, 0), (3, 2, 1)]

itertools.product 产生多个列表和迭代器的(积)

>>> x = itertools.product('ABC', range(3))
>>>
>>> print(list(x))
[('A', 0), ('A', 1), ('A', 2), ('B', 0), ('B', 1), ('B', 2), ('C', 0), ('C', 1), ('C', 2)]

itertools.repeat 简单的生成一个拥有指定数目元素的迭代器

>>> x = itertools.repeat(0, 5)
>>> print(list(x))
[0, 0, 0, 0, 0]

itertools.starmap 类似map

>>> x = itertools.starmap(str.islower, 'aBCDefGhI')
>>> print(list(x))
[True, False, False, False, True, True, False, True, False]

itertools.takewhile 与dropwhile相反,保留元素直至真值函数值为假。

>>> x = itertools.takewhile(lambda e: e < 5, range(10))
>>> print(list(x))
[0, 1, 2, 3, 4]

itertools.zip_longest 类似于zip,不过已较长的列表和迭代器的长度为准

>>> x = itertools.zip_longest(range(3), range(5))
>>> y = zip(range(3), range(5))
>>> print(list(x))
[(0, 0), (1, 1), (2, 2), (None, 3), (None, 4)]
>>> print(list(y))
[(0, 0), (1, 1), (2, 2)]
7.推导式 Comprehension ,可迭代对象 Iterable,迭代器 Iterator 及 生成器 Generators

7.1 推导式

推导式一般用的最多的是列表推导式(list comprehensions)

example:a = [i*i for i in range(10)]

官方解释:列表解析式是Python内置的非常简单却强大的可以用来创建list的生成式,在一个中括号里包含一个表达式, 然后是一个for语句, 然后是0个或多个for或者if语句。 那个表达式可以是任意的, 意思是你可以在列表中放任意类型的对象。返回结果将是一个新的列表,在这个以if和for语句为上下文的表达式运行完成之后产生。

可以看到,使用列表解析式的写法更加简短,除此之外,因为是Python内置的用法,底层使用c语言实现,相较于编写Python代码而言,运行速度更快。

====总结====:如果要对现有可迭代对象做一下处理,然后生成新的列表,使用列表推导式将是最便捷的方法。

7.2 可迭代对象 Iterable

最简单的解释:可以使用for...in...语句进行循环的对象,就是可迭代对象(Iterable),可以使用isinstance()方法进行判断。list、dict、str是Iterable(可迭代对象),但不是Iterator(迭代器),可以使用iter()函数把list、dict、str等Iterable变成Iterator。

temp=iter([1, 2, 3]) print type(temp) # list_iterator print next(temp) # 1

7.3 迭代器 Iterator

任意对象,只要定义了next (Python2) 或者next方法并实现了iter方法,它就是一个迭代器。 迭代器的优势:在构建迭代器时,不是将所有元素一次性的加载,而是等调用next方法时返回元素,大量节省内存。 迭代器应用场景:数列的数据规模巨大且数列有规律,但是不能使用列表推导式描述的时候,如大数据量的数列的创建,大文件的读取等。

7.4 生成器 Generators

生成器是一个特殊的函数,可以被用作控制循环的迭代行为,python中生成器是一种高级迭代器,使用yield代替return返回,每次调用yield会暂停,而可以使用next()函数和send()函数恢复生成器。 生成器类似于返回值为一个数组的一个函数,这个函数可以接收参数,可以被调用,但是不同于一般的函数会一次性返回包括了所有数值的数组,生成器一次只能产生一个值,这样消耗的内存数量将大大减小,生成器可以被看作是一个函数,但是它本质上也算是一个迭代器。

生成器函数:为什么叫生成器函数?因为它随着时间的推移生成了一个数值队列。一般的函数在执行完毕之后会返回一个值然后退出,但是生成器函数会自动挂起,然后重新拾起急需执行,利用yield关键字关起函数,给调用者返回一个值,同时保留了当前的足够多的状态,可以使函数继续执行。当需要一个将返回一个序列或在循环中执行的函数时,就可以使用生成器,因为当这些元素被传递到另一个函数中进行后续处理时,一次返回一个元素可以有效的提升整体性能。常见的应用场景是使用生成器生成数据流缓冲区。

生成器表达式:生成式表达式是一种实现生成器的便捷方式,将列表推导式的中括号替换为圆括号。返回一个对象,这个对象只有在需要的时候才产生结果。 生成器表达式的一个例子: g = (xx for x in range(10))

[参考文章]: python 生成器和迭代器有这篇就够了 - 战争热诚 - 博客园 https://www.cnblogs.com/wj-1314/p/8490822.html 迭代器 生成器, 可迭代对象以及应用场景 - 若无过客丶何来人生 - 博客园 https://www.cnblogs.com/liurenli/p/10127818.html

8.命名切片

slice() 函数 描述:slice() 函数用实现切片对象,主要在切片操作函数里的参数传递。 slice 语法: class slice(stop) class slice(start, stop[, step])

如果你有一个切片对象 a,你可以分别调用它的 a.start , a.stop , a.step 属性来获取更多的信息。 另外,你还能通过调用切片的 indices(size) 方法将它映射到一个确定大小的序列上,这个方法返回一个三元组 (start, stop, step) ,所有值都会被合适的缩小以满足边界限制,从而使用的时候避免出现 IndexError 异常。

文档的说明: class slice(object) | slice(stop) | slice(start, stop[, step]) | | Create a slice object. This is used for extended slicing (e.g. a[0:10:2]). | | Methods defined here: | | eq(self, value, /) | Return self==value. | | ge(self, value, /) | Return self>=value. | | getattribute(self, name, /) | Return getattr(self, name). | | gt(self, value, /) | Return self>value. | | le(self, value, /) | Return self<=value. | | lt(self, value, /) | Return self<value. | | ne(self, value, /) | Return self!=value. | | reduce(...) | Return state information for pickling. | | repr(self, /) | Return repr(self). | | indices(...) | S.indices(len) -> (start, stop, stride) | | Assuming a sequence of length len, calculate the start and stop | indices, and the stride length of the extended slice described by | S. Out of bounds indices are clipped in a manner consistent with the | handling of normal slices. | | ---------------------------------------------------------------------- | Static methods defined here: | | new(*args, **kwargs) from builtins.type | Create and return a new object. See help(type) for accurate signature.-- More -- | | ---------------------------------------------------------------------- | Data descriptors defined here: | | start | | step | | stop | | ---------------------------------------------------------------------- | Data and other attributes defined here: | | hash = None

9.上下文管理器:

常见用法: with open('test.txt') as f: print f.readlines()

几个概念

  • 上下文表达式:with open('test.txt') as f:
  • 上下文管理器:open('test.txt')
  • f 不是上下文管理器,应该是资源对象。 简单点说,就是在一个类里,实现了enterexit的方法,这个类的实例就是一个上下文管理器。

为什么要使用上下文管理器?

  • 可以以一种更加优雅的方式,操作(创建/获取/释放)资源,如文件操作、数据库连接;
  • 可以以一种更加优雅的方式,处理异常;

一个基于类实现的上下文理器: class File(object): def init(self, file_name, method): self.file_obj = open(file_name, method) def enter(self): return self.file_obj def exit(self, exc_type, exc_value, exc_traceback): self.file_obj.close()

注:exc_type:异常类型 , exc_value:异常值, exc_traceback:异常的错误栈信息

执行过程:

with语句先暂存了File类的exit方法; 然后它调用File类的enter方法; enter方法打开文件并返回给with语句; 打开的文件句柄被传递给opened_file参数; 我们使用.write()来写文件; with语句调用之前暂存的exit方法; exit方法关闭了文件。

[关于上下文管理器的更多详细可以参考文章]: 浅淡python中with的用法,上下文管理器 - 听风。 - 博客园 https://www.cnblogs.com/huchong/p/8268765.html 深入理解 Python 中的上下文管理器 - 王一白 - 博客园 https://www.cnblogs.com/wongbingming/p/10519553.html

统一声明:关于原创博客内容,可能会有部分内容参考自互联网,如有原创链接会声明引用;如找不到原创链接,在此声明如有侵权请联系删除哈。关于转载博客,如有原创链接会声明;如找不到原创链接,如果有哪些问题、有些谬误、有哪些不妥或者侵犯到您的权益的地方,可以联系我,我马上修改。