1 :mod:`nntplib` --- NNTP protocol client 2 ======================================= 3 4 .. module:: nntplib 5 :synopsis: NNTP protocol client (requires sockets). 6 7 **Source code:** :source:`Lib/nntplib.py` 8 9 .. index:: 10 pair: NNTP; protocol 11 single: Network News Transfer Protocol 12 13 -------------- 14 15 This module defines the class :class:`NNTP` which implements the client side of 16 the Network News Transfer Protocol. It can be used to implement a news reader 17 or poster, or automated news processors. It is compatible with :rfc:`3977` 18 as well as the older :rfc:`977` and :rfc:`2980`. 19 20 Here are two small examples of how it can be used. To list some statistics 21 about a newsgroup and print the subjects of the last 10 articles:: 22 23 >>> s = nntplib.NNTP('news.gmane.org') 24 >>> resp, count, first, last, name = s.group('gmane.comp.python.committers') 25 >>> print('Group', name, 'has', count, 'articles, range', first, 'to', last) 26 Group gmane.comp.python.committers has 1096 articles, range 1 to 1096 27 >>> resp, overviews = s.over((last - 9, last)) 28 >>> for id, over in overviews: 29 ... print(id, nntplib.decode_header(over['subject'])) 30 ... 31 1087 Re: Commit privileges for ukasz Langa 32 1088 Re: 3.2 alpha 2 freeze 33 1089 Re: 3.2 alpha 2 freeze 34 1090 Re: Commit privileges for ukasz Langa 35 1091 Re: Commit privileges for ukasz Langa 36 1092 Updated ssh key 37 1093 Re: Updated ssh key 38 1094 Re: Updated ssh key 39 1095 Hello fellow committers! 40 1096 Re: Hello fellow committers! 41 >>> s.quit() 42 '205 Bye!' 43 44 To post an article from a binary file (this assumes that the article has valid 45 headers, and that you have right to post on the particular newsgroup):: 46 47 >>> s = nntplib.NNTP('news.gmane.org') 48 >>> f = open('article.txt', 'rb') 49 >>> s.post(f) 50 '240 Article posted successfully.' 51 >>> s.quit() 52 '205 Bye!' 53 54 The module itself defines the following classes: 55 56 57 .. class:: NNTP(host, port=119, user=None, password=None, readermode=None, usenetrc=False, [timeout]) 58 59 Return a new :class:`NNTP` object, representing a connection 60 to the NNTP server running on host *host*, listening at port *port*. 61 An optional *timeout* can be specified for the socket connection. 62 If the optional *user* and *password* are provided, or if suitable 63 credentials are present in :file:`/.netrc` and the optional flag *usenetrc* 64 is true, the ``AUTHINFO USER`` and ``AUTHINFO PASS`` commands are used 65 to identify and authenticate the user to the server. If the optional 66 flag *readermode* is true, then a ``mode reader`` command is sent before 67 authentication is performed. Reader mode is sometimes necessary if you are 68 connecting to an NNTP server on the local machine and intend to call 69 reader-specific commands, such as ``group``. If you get unexpected 70 :exc:`NNTPPermanentError`\ s, you might need to set *readermode*. 71 The :class:`NNTP` class supports the :keyword:`with` statement to 72 unconditionally consume :exc:`OSError` exceptions and to close the NNTP 73 connection when done, e.g.: 74 75 >>> from nntplib import NNTP 76 >>> with NNTP('news.gmane.org') as n: 77 ... n.group('gmane.comp.python.committers') 78 ... # doctest: +SKIP 79 ('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers') 80 >>> 81 82 83 .. versionchanged:: 3.2 84 *usenetrc* is now ``False`` by default. 85 86 .. versionchanged:: 3.3 87 Support for the :keyword:`with` statement was added. 88 89 .. class:: NNTP_SSL(host, port=563, user=None, password=None, ssl_context=None, readermode=None, usenetrc=False, [timeout]) 90 91 Return a new :class:`NNTP_SSL` object, representing an encrypted 92 connection to the NNTP server running on host *host*, listening at 93 port *port*. :class:`NNTP_SSL` objects have the same methods as 94 :class:`NNTP` objects. If *port* is omitted, port 563 (NNTPS) is used. 95 *ssl_context* is also optional, and is a :class:`~ssl.SSLContext` object. 96 Please read :ref:`ssl-security` for best practices. 97 All other parameters behave the same as for :class:`NNTP`. 98 99 Note that SSL-on-563 is discouraged per :rfc:`4642`, in favor of 100 STARTTLS as described below. However, some servers only support the 101 former. 102 103 .. versionadded:: 3.2 104 105 .. versionchanged:: 3.4 106 The class now supports hostname check with 107 :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see 108 :data:`ssl.HAS_SNI`). 109 110 .. exception:: NNTPError 111 112 Derived from the standard exception :exc:`Exception`, this is the base 113 class for all exceptions raised by the :mod:`nntplib` module. Instances 114 of this class have the following attribute: 115 116 .. attribute:: response 117 118 The response of the server if available, as a :class:`str` object. 119 120 121 .. exception:: NNTPReplyError 122 123 Exception raised when an unexpected reply is received from the server. 124 125 126 .. exception:: NNTPTemporaryError 127 128 Exception raised when a response code in the range 400--499 is received. 129 130 131 .. exception:: NNTPPermanentError 132 133 Exception raised when a response code in the range 500--599 is received. 134 135 136 .. exception:: NNTPProtocolError 137 138 Exception raised when a reply is received from the server that does not begin 139 with a digit in the range 1--5. 140 141 142 .. exception:: NNTPDataError 143 144 Exception raised when there is some error in the response data. 145 146 147 .. _nntp-objects: 148 149 NNTP Objects 150 ------------ 151 152 When connected, :class:`NNTP` and :class:`NNTP_SSL` objects support the 153 following methods and attributes. 154 155 Attributes 156 ^^^^^^^^^^ 157 158 .. attribute:: NNTP.nntp_version 159 160 An integer representing the version of the NNTP protocol supported by the 161 server. In practice, this should be ``2`` for servers advertising 162 :rfc:`3977` compliance and ``1`` for others. 163 164 .. versionadded:: 3.2 165 166 .. attribute:: NNTP.nntp_implementation 167 168 A string describing the software name and version of the NNTP server, 169 or :const:`None` if not advertised by the server. 170 171 .. versionadded:: 3.2 172 173 Methods 174 ^^^^^^^ 175 176 The *response* that is returned as the first item in the return tuple of almost 177 all methods is the server's response: a string beginning with a three-digit 178 code. If the server's response indicates an error, the method raises one of 179 the above exceptions. 180 181 Many of the following methods take an optional keyword-only argument *file*. 182 When the *file* argument is supplied, it must be either a :term:`file object` 183 opened for binary writing, or the name of an on-disk file to be written to. 184 The method will then write any data returned by the server (except for the 185 response line and the terminating dot) to the file; any list of lines, 186 tuples or objects that the method normally returns will be empty. 187 188 .. versionchanged:: 3.2 189 Many of the following methods have been reworked and fixed, which makes 190 them incompatible with their 3.1 counterparts. 191 192 193 .. method:: NNTP.quit() 194 195 Send a ``QUIT`` command and close the connection. Once this method has been 196 called, no other methods of the NNTP object should be called. 197 198 199 .. method:: NNTP.getwelcome() 200 201 Return the welcome message sent by the server in reply to the initial 202 connection. (This message sometimes contains disclaimers or help information 203 that may be relevant to the user.) 204 205 206 .. method:: NNTP.getcapabilities() 207 208 Return the :rfc:`3977` capabilities advertised by the server, as a 209 :class:`dict` instance mapping capability names to (possibly empty) lists 210 of values. On legacy servers which don't understand the ``CAPABILITIES`` 211 command, an empty dictionary is returned instead. 212 213 >>> s = NNTP('news.gmane.org') 214 >>> 'POST' in s.getcapabilities() 215 True 216 217 .. versionadded:: 3.2 218 219 220 .. method:: NNTP.login(user=None, password=None, usenetrc=True) 221 222 Send ``AUTHINFO`` commands with the user name and password. If *user* 223 and *password* are ``None`` and *usenetrc* is true, credentials from 224 ``~/.netrc`` will be used if possible. 225 226 Unless intentionally delayed, login is normally performed during the 227 :class:`NNTP` object initialization and separately calling this function 228 is unnecessary. To force authentication to be delayed, you must not set 229 *user* or *password* when creating the object, and must set *usenetrc* to 230 False. 231 232 .. versionadded:: 3.2 233 234 235 .. method:: NNTP.starttls(ssl_context=None) 236 237 Send a ``STARTTLS`` command. This will enable encryption on the NNTP 238 connection. The *ssl_context* argument is optional and should be a 239 :class:`ssl.SSLContext` object. Please read :ref:`ssl-security` for best 240 practices. 241 242 Note that this may not be done after authentication information has 243 been transmitted, and authentication occurs by default if possible during a 244 :class:`NNTP` object initialization. See :meth:`NNTP.login` for information 245 on suppressing this behavior. 246 247 .. versionadded:: 3.2 248 249 .. versionchanged:: 3.4 250 The method now supports hostname check with 251 :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see 252 :data:`ssl.HAS_SNI`). 253 254 .. method:: NNTP.newgroups(date, *, file=None) 255 256 Send a ``NEWGROUPS`` command. The *date* argument should be a 257 :class:`datetime.date` or :class:`datetime.datetime` object. 258 Return a pair ``(response, groups)`` where *groups* is a list representing 259 the groups that are new since the given *date*. If *file* is supplied, 260 though, then *groups* will be empty. 261 262 >>> from datetime import date, timedelta 263 >>> resp, groups = s.newgroups(date.today() - timedelta(days=3)) 264 >>> len(groups) # doctest: +SKIP 265 85 266 >>> groups[0] # doctest: +SKIP 267 GroupInfo(group='gmane.network.tor.devel', last='4', first='1', flag='m') 268 269 270 .. method:: NNTP.newnews(group, date, *, file=None) 271 272 Send a ``NEWNEWS`` command. Here, *group* is a group name or ``'*'``, and 273 *date* has the same meaning as for :meth:`newgroups`. Return a pair 274 ``(response, articles)`` where *articles* is a list of message ids. 275 276 This command is frequently disabled by NNTP server administrators. 277 278 279 .. method:: NNTP.list(group_pattern=None, *, file=None) 280 281 Send a ``LIST`` or ``LIST ACTIVE`` command. Return a pair 282 ``(response, list)`` where *list* is a list of tuples representing all 283 the groups available from this NNTP server, optionally matching the 284 pattern string *group_pattern*. Each tuple has the form 285 ``(group, last, first, flag)``, where *group* is a group name, *last* 286 and *first* are the last and first article numbers, and *flag* usually 287 takes one of these values: 288 289 * ``y``: Local postings and articles from peers are allowed. 290 * ``m``: The group is moderated and all postings must be approved. 291 * ``n``: No local postings are allowed, only articles from peers. 292 * ``j``: Articles from peers are filed in the junk group instead. 293 * ``x``: No local postings, and articles from peers are ignored. 294 * ``=foo.bar``: Articles are filed in the ``foo.bar`` group instead. 295 296 If *flag* has another value, then the status of the newsgroup should be 297 considered unknown. 298 299 This command can return very large results, especially if *group_pattern* 300 is not specified. It is best to cache the results offline unless you 301 really need to refresh them. 302 303 .. versionchanged:: 3.2 304 *group_pattern* was added. 305 306 307 .. method:: NNTP.descriptions(grouppattern) 308 309 Send a ``LIST NEWSGROUPS`` command, where *grouppattern* is a wildmat string as 310 specified in :rfc:`3977` (it's essentially the same as DOS or UNIX shell wildcard 311 strings). Return a pair ``(response, descriptions)``, where *descriptions* 312 is a dictionary mapping group names to textual descriptions. 313 314 >>> resp, descs = s.descriptions('gmane.comp.python.*') 315 >>> len(descs) # doctest: +SKIP 316 295 317 >>> descs.popitem() # doctest: +SKIP 318 ('gmane.comp.python.bio.general', 'BioPython discussion list (Moderated)') 319 320 321 .. method:: NNTP.description(group) 322 323 Get a description for a single group *group*. If more than one group matches 324 (if 'group' is a real wildmat string), return the first match. If no group 325 matches, return an empty string. 326 327 This elides the response code from the server. If the response code is needed, 328 use :meth:`descriptions`. 329 330 331 .. method:: NNTP.group(name) 332 333 Send a ``GROUP`` command, where *name* is the group name. The group is 334 selected as the current group, if it exists. Return a tuple 335 ``(response, count, first, last, name)`` where *count* is the (estimated) 336 number of articles in the group, *first* is the first article number in 337 the group, *last* is the last article number in the group, and *name* 338 is the group name. 339 340 341 .. method:: NNTP.over(message_spec, *, file=None) 342 343 Send an ``OVER`` command, or an ``XOVER`` command on legacy servers. 344 *message_spec* can be either a string representing a message id, or 345 a ``(first, last)`` tuple of numbers indicating a range of articles in 346 the current group, or a ``(first, None)`` tuple indicating a range of 347 articles starting from *first* to the last article in the current group, 348 or :const:`None` to select the current article in the current group. 349 350 Return a pair ``(response, overviews)``. *overviews* is a list of 351 ``(article_number, overview)`` tuples, one for each article selected 352 by *message_spec*. Each *overview* is a dictionary with the same number 353 of items, but this number depends on the server. These items are either 354 message headers (the key is then the lower-cased header name) or metadata 355 items (the key is then the metadata name prepended with ``":"``). The 356 following items are guaranteed to be present by the NNTP specification: 357 358 * the ``subject``, ``from``, ``date``, ``message-id`` and ``references`` 359 headers 360 * the ``:bytes`` metadata: the number of bytes in the entire raw article 361 (including headers and body) 362 * the ``:lines`` metadata: the number of lines in the article body 363 364 The value of each item is either a string, or :const:`None` if not present. 365 366 It is advisable to use the :func:`decode_header` function on header 367 values when they may contain non-ASCII characters:: 368 369 >>> _, _, first, last, _ = s.group('gmane.comp.python.devel') 370 >>> resp, overviews = s.over((last, last)) 371 >>> art_num, over = overviews[0] 372 >>> art_num 373 117216 374 >>> list(over.keys()) 375 ['xref', 'from', ':lines', ':bytes', 'references', 'date', 'message-id', 'subject'] 376 >>> over['from'] 377 '=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?= <martin (a] v.loewis.de>' 378 >>> nntplib.decode_header(over['from']) 379 '"Martin v. Lwis" <martin (a] v.loewis.de>' 380 381 .. versionadded:: 3.2 382 383 384 .. method:: NNTP.help(*, file=None) 385 386 Send a ``HELP`` command. Return a pair ``(response, list)`` where *list* is a 387 list of help strings. 388 389 390 .. method:: NNTP.stat(message_spec=None) 391 392 Send a ``STAT`` command, where *message_spec* is either a message id 393 (enclosed in ``'<'`` and ``'>'``) or an article number in the current group. 394 If *message_spec* is omitted or :const:`None`, the current article in the 395 current group is considered. Return a triple ``(response, number, id)`` 396 where *number* is the article number and *id* is the message id. 397 398 >>> _, _, first, last, _ = s.group('gmane.comp.python.devel') 399 >>> resp, number, message_id = s.stat(first) 400 >>> number, message_id 401 (9099, '<20030112190404.GE29873 (a] epoch.metaslash.com>') 402 403 404 .. method:: NNTP.next() 405 406 Send a ``NEXT`` command. Return as for :meth:`.stat`. 407 408 409 .. method:: NNTP.last() 410 411 Send a ``LAST`` command. Return as for :meth:`.stat`. 412 413 414 .. method:: NNTP.article(message_spec=None, *, file=None) 415 416 Send an ``ARTICLE`` command, where *message_spec* has the same meaning as 417 for :meth:`.stat`. Return a tuple ``(response, info)`` where *info* 418 is a :class:`~collections.namedtuple` with three attributes *number*, 419 *message_id* and *lines* (in that order). *number* is the article number 420 in the group (or 0 if the information is not available), *message_id* the 421 message id as a string, and *lines* a list of lines (without terminating 422 newlines) comprising the raw message including headers and body. 423 424 >>> resp, info = s.article('<20030112190404.GE29873 (a] epoch.metaslash.com>') 425 >>> info.number 426 0 427 >>> info.message_id 428 '<20030112190404.GE29873 (a] epoch.metaslash.com>' 429 >>> len(info.lines) 430 65 431 >>> info.lines[0] 432 b'Path: main.gmane.org!not-for-mail' 433 >>> info.lines[1] 434 b'From: Neal Norwitz <neal (a] metaslash.com>' 435 >>> info.lines[-3:] 436 [b'There is a patch for 2.3 as well as 2.2.', b'', b'Neal'] 437 438 439 .. method:: NNTP.head(message_spec=None, *, file=None) 440 441 Same as :meth:`article()`, but sends a ``HEAD`` command. The *lines* 442 returned (or written to *file*) will only contain the message headers, not 443 the body. 444 445 446 .. method:: NNTP.body(message_spec=None, *, file=None) 447 448 Same as :meth:`article()`, but sends a ``BODY`` command. The *lines* 449 returned (or written to *file*) will only contain the message body, not the 450 headers. 451 452 453 .. method:: NNTP.post(data) 454 455 Post an article using the ``POST`` command. The *data* argument is either 456 a :term:`file object` opened for binary reading, or any iterable of bytes 457 objects (representing raw lines of the article to be posted). It should 458 represent a well-formed news article, including the required headers. The 459 :meth:`post` method automatically escapes lines beginning with ``.`` and 460 appends the termination line. 461 462 If the method succeeds, the server's response is returned. If the server 463 refuses posting, a :class:`NNTPReplyError` is raised. 464 465 466 .. method:: NNTP.ihave(message_id, data) 467 468 Send an ``IHAVE`` command. *message_id* is the id of the message to send 469 to the server (enclosed in ``'<'`` and ``'>'``). The *data* parameter 470 and the return value are the same as for :meth:`post()`. 471 472 473 .. method:: NNTP.date() 474 475 Return a pair ``(response, date)``. *date* is a :class:`~datetime.datetime` 476 object containing the current date and time of the server. 477 478 479 .. method:: NNTP.slave() 480 481 Send a ``SLAVE`` command. Return the server's *response*. 482 483 484 .. method:: NNTP.set_debuglevel(level) 485 486 Set the instance's debugging level. This controls the amount of debugging 487 output printed. The default, ``0``, produces no debugging output. A value of 488 ``1`` produces a moderate amount of debugging output, generally a single line 489 per request or response. A value of ``2`` or higher produces the maximum amount 490 of debugging output, logging each line sent and received on the connection 491 (including message text). 492 493 494 The following are optional NNTP extensions defined in :rfc:`2980`. Some of 495 them have been superseded by newer commands in :rfc:`3977`. 496 497 498 .. method:: NNTP.xhdr(hdr, str, *, file=None) 499 500 Send an ``XHDR`` command. The *hdr* argument is a header keyword, e.g. 501 ``'subject'``. The *str* argument should have the form ``'first-last'`` 502 where *first* and *last* are the first and last article numbers to search. 503 Return a pair ``(response, list)``, where *list* is a list of pairs ``(id, 504 text)``, where *id* is an article number (as a string) and *text* is the text of 505 the requested header for that article. If the *file* parameter is supplied, then 506 the output of the ``XHDR`` command is stored in a file. If *file* is a string, 507 then the method will open a file with that name, write to it then close it. 508 If *file* is a :term:`file object`, then it will start calling :meth:`write` on 509 it to store the lines of the command output. If *file* is supplied, then the 510 returned *list* is an empty list. 511 512 513 .. method:: NNTP.xover(start, end, *, file=None) 514 515 Send an ``XOVER`` command. *start* and *end* are article numbers 516 delimiting the range of articles to select. The return value is the 517 same of for :meth:`over()`. It is recommended to use :meth:`over()` 518 instead, since it will automatically use the newer ``OVER`` command 519 if available. 520 521 522 .. method:: NNTP.xpath(id) 523 524 Return a pair ``(resp, path)``, where *path* is the directory path to the 525 article with message ID *id*. Most of the time, this extension is not 526 enabled by NNTP server administrators. 527 528 .. deprecated:: 3.3 529 The XPATH extension is not actively used. 530 531 532 .. XXX deprecated: 533 534 .. method:: NNTP.xgtitle(name, *, file=None) 535 536 Process an ``XGTITLE`` command, returning a pair ``(response, list)``, where 537 *list* is a list of tuples containing ``(name, title)``. If the *file* parameter 538 is supplied, then the output of the ``XGTITLE`` command is stored in a file. 539 If *file* is a string, then the method will open a file with that name, write 540 to it then close it. If *file* is a :term:`file object`, then it will start 541 calling :meth:`write` on it to store the lines of the command output. If *file* 542 is supplied, then the returned *list* is an empty list. This is an optional NNTP 543 extension, and may not be supported by all servers. 544 545 RFC2980 says "It is suggested that this extension be deprecated". Use 546 :meth:`descriptions` or :meth:`description` instead. 547 548 549 Utility functions 550 ----------------- 551 552 The module also defines the following utility function: 553 554 555 .. function:: decode_header(header_str) 556 557 Decode a header value, un-escaping any escaped non-ASCII characters. 558 *header_str* must be a :class:`str` object. The unescaped value is 559 returned. Using this function is recommended to display some headers 560 in a human readable form:: 561 562 >>> decode_header("Some subject") 563 'Some subject' 564 >>> decode_header("=?ISO-8859-15?Q?D=E9buter_en_Python?=") 565 'Dbuter en Python' 566 >>> decode_header("Re: =?UTF-8?B?cHJvYmzDqG1lIGRlIG1hdHJpY2U=?=") 567 'Re: problme de matrice' 568
