{ "cells": [ { "cell_type": "markdown", "id": "absent-economy", "metadata": {}, "source": [ "# Posting from Python Code\n", "\n", "If you have some python code that is already creating files, and\n", "you have no wish to invoke a separate program to post the files,\n", "one can access message posting logic easily, given an existing file.\n", "\n", "This example is for programmatically creating posts for files.\n", "It requires write access to a broker, with a user that is allowed\n", "to post to an exchange. \n", "\n", "Need to establish a posting configuration, and then instantiate a posting_engine (an instance than can be used to post messages.)" ] }, { "cell_type": "code", "execution_count": 1, "id": "prescribed-mortgage", "metadata": { "scrolled": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-05-27 11:02:29,889 [DEBUG] amqp _on_start Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'basic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct_reply_to': True}, 'cluster_name': 'rabbit@fractal', 'copyright': 'Copyright (c) 2007-2022 VMware, Inc. or its affiliates.', 'information': 'Licensed under the MPL 2.0. Website: https://rabbitmq.com', 'platform': 'Erlang/OTP 24.2.1', 'product': 'RabbitMQ', 'version': '3.9.13'}, mechanisms: [b'PLAIN', b'AMQPLAIN'], locales: ['en_US']\n", "2023-05-27 11:02:29,890 [DEBUG] amqp __init__ using channel_id: 1\n", "2023-05-27 11:02:29,890 [DEBUG] amqp _on_open_ok Channel open\n", "2023-05-27 11:02:29,891 [DEBUG] sarracenia.moth.amqp __putSetup putSetup ... 1. declaring xpublic\n", "2023-05-27 11:02:29,892 [INFO] sarracenia.moth.amqp __putSetup exchange declared: xpublic (as: amqp://tfeed@localhost)\n", "2023-05-27 11:02:29,892 [DEBUG] sarracenia.moth.amqp __putSetup putSetup ... Done!\n" ] } ], "source": [ "import sarracenia\n", "import sarracenia.moth\n", "import sarracenia.credentials\n", "from sarracenia.config import default_config\n", "\n", "import os\n", "import time\n", "import socket\n", "\n", "cfg = default_config()\n", "cfg.logLevel = 'debug'\n", "cfg.broker = sarracenia.credentials.Credential('amqp://tfeed:password@localhost')\n", "cfg.exchange = 'xpublic'\n", "cfg.post_baseUrl = 'http://host'\n", "cfg.post_baseDir = '/tmp'\n", "\n", "# moth wants a dict as options, rather than sarracenia.config.Config instance.\n", "posting_engine = sarracenia.moth.Moth.pubFactory( cfg.dictify() )" ] }, { "cell_type": "markdown", "id": "pharmaceutical-airport", "metadata": {}, "source": [ "next we create a text file... " ] }, { "cell_type": "code", "execution_count": 2, "id": "early-background", "metadata": {}, "outputs": [], "source": [ "sample_fileName = '/tmp/sample.txt'\n", "sample_file = open( sample_fileName , 'w')\n", "sample_file.write(\n", "\"\"\"\n", "CACN00 CWAO 161800\n", "PMN\n", "160,2021,228,1800,1065,100,-6999,20.49,43.63,16.87,16.64,323.5,9.32,27.31,1740,317.8,19.22,1.609,230.7,230.7,230.7,230.7,0,0,0,16.38,15.59,305.\n", "9,17.8,16.38,19.35,55.66,15.23,14.59,304,16.67,3.844,20.51,18.16,0,0,-6999,-6999,-6999,-6999,-6999,-6999,-6999,-6999,0,0,0,0,0,0,0,0,0,0,0,0,0,\n", "13.41,13.85,27.07,3473\n", "\"\"\"\n", ")\n", "sample_file.close()" ] }, { "cell_type": "markdown", "id": "affected-oxygen", "metadata": {}, "source": [ "you give the file name, the config initialized above, and the stat record for the file to the msg_init() function. It will return a message that is ready to feed to the posting_engine." ] }, { "cell_type": "code", "execution_count": 3, "id": "precise-delivery", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-05-27 11:02:41,866 [DEBUG] sarracenia __computeIdentity xattr sum too old\n", "2023-05-27 11:02:41,868 [DEBUG] sarracenia.moth.amqp putNewMessage published body: {\"pubTime\": \"20230527T150241.865911961\", \"relPath\": \"sample.txt\", \"baseUrl\": \"http://host\", \"mode\": \"664\", \"size\": 335, \"mtime\": \"20230527T150237.927556038\", \"atime\": \"20230525T031938.0635721684\", \"identity\": {\"method\": \"sha512\", \"value\": \"w5ZwUT1IMAjnQT6TLR9NSLzG5RKijhxq46FjMx5UWtsHM/FNOaYNRmGwonIPfnhE5xUORf3z5dRyI6zdL6ygNw==\"}} headers: {} to xpublic under: v03 \n", "2023-05-27 11:02:41,868 [DEBUG] amqp collect Closed channel #1\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "{'_format': 'v03', '_deleteOnPost': {'local_offset', 'new_baseUrl', 'new_dir', 'new_subtopic', 'new_file', 'subtopic', '_format', 'exchange', 'new_relPath', 'post_format'}, 'exchange': 'xpublic', 'local_offset': 0, 'pubTime': '20230527T150241.865911961', 'new_dir': '/tmp', 'new_file': 'sample.txt', 'post_format': 'v03', 'new_baseUrl': 'http://host', 'new_relPath': 'sample.txt', 'new_subtopic': [], 'relPath': 'sample.txt', 'subtopic': [], 'baseUrl': 'http://host', 'mode': '664', 'size': 335, 'mtime': '20230527T150237.927556038', 'atime': '20230525T031938.0635721684', 'identity': {'method': 'sha512', 'value': 'w5ZwUT1IMAjnQT6TLR9NSLzG5RKijhxq46FjMx5UWtsHM/FNOaYNRmGwonIPfnhE5xUORf3z5dRyI6zdL6ygNw=='}}\n" ] } ], "source": [ "# you can supply msg_init with your files, it will build a message appropriate for it.\n", "m = sarracenia.Message.fromFileData(sample_fileName, cfg, os.stat(sample_fileName) )\n", "# here is the resulting message.\n", "print(m)\n", "\n", "# feed the message to the posting engine.\n", "posting_engine.putNewMessage(m)\n", "\n", "# when done, should close... cleaner...\n", "posting_engine.close() " ] }, { "cell_type": "markdown", "id": "cleared-worst", "metadata": {}, "source": [ "One can post as many messages as needed with putNewMessage, when finished with posting files, to shut down the connection with the broker cleanly, please close the posting_engine." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.6" } }, "nbformat": 4, "nbformat_minor": 5 }