Posting from Python Code

If you have some python code that is already creating files, and you have no wish to invoke a separate program to post the files, one can access message posting logic easily, given an existing file.

This example is for programmatically creating posts for files. It requires write access to a broker, with a user that is allowed to post to an exchange.

Need to establish a posting configuration, and then instantiate a posting_engine (an instance than can be used to post messages.)

[1]:

import sarracenia
import sarracenia.moth
import sarracenia.credentials
from sarracenia.config import default_config

import os
import time
import socket

cfg = default_config()
cfg.logLevel = 'debug'
cfg.broker = sarracenia.credentials.Credential('amqp://tfeed:password@localhost')
cfg.exchange = 'xpublic'
cfg.post_baseUrl = 'http://host'
cfg.post_baseDir = '/tmp'

# moth wants a dict as options, rather than sarracenia.config.Config instance.
posting_engine = sarracenia.moth.Moth.pubFactory( cfg.dictify() )

2023-05-27 11:02:29,889 [DEBUG] amqp _on_start Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'basic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct_reply_to': True}, 'cluster_name': 'rabbit@fractal', 'copyright': 'Copyright (c) 2007-2022 VMware, Inc. or its affiliates.', 'information': 'Licensed under the MPL 2.0. Website: https://rabbitmq.com', 'platform': 'Erlang/OTP 24.2.1', 'product': 'RabbitMQ', 'version': '3.9.13'}, mechanisms: [b'PLAIN', b'AMQPLAIN'], locales: ['en_US']
2023-05-27 11:02:29,890 [DEBUG] amqp __init__ using channel_id: 1
2023-05-27 11:02:29,890 [DEBUG] amqp _on_open_ok Channel open
2023-05-27 11:02:29,891 [DEBUG] sarracenia.moth.amqp __putSetup putSetup ... 1. declaring xpublic
2023-05-27 11:02:29,892 [INFO] sarracenia.moth.amqp __putSetup exchange declared: xpublic (as: amqp://tfeed@localhost)
2023-05-27 11:02:29,892 [DEBUG] sarracenia.moth.amqp __putSetup putSetup ... Done!

next we create a text file…

[2]:

sample_fileName = '/tmp/sample.txt'
sample_file = open( sample_fileName , 'w')
sample_file.write(
"""
CACN00 CWAO 161800
PMN
160,2021,228,1800,1065,100,-6999,20.49,43.63,16.87,16.64,323.5,9.32,27.31,1740,317.8,19.22,1.609,230.7,230.7,230.7,230.7,0,0,0,16.38,15.59,305.
9,17.8,16.38,19.35,55.66,15.23,14.59,304,16.67,3.844,20.51,18.16,0,0,-6999,-6999,-6999,-6999,-6999,-6999,-6999,-6999,0,0,0,0,0,0,0,0,0,0,0,0,0,
13.41,13.85,27.07,3473
"""
)
sample_file.close()

you give the file name, the config initialized above, and the stat record for the file to the msg_init() function. It will return a message that is ready to feed to the posting_engine.

[3]:

# you can supply msg_init with your files, it will build a message appropriate for it.
m = sarracenia.Message.fromFileData(sample_fileName, cfg, os.stat(sample_fileName) )
# here is the resulting message.
print(m)

# feed the message to the posting engine.
posting_engine.putNewMessage(m)

# when done, should close... cleaner...
posting_engine.close()

2023-05-27 11:02:41,866 [DEBUG] sarracenia __computeIdentity xattr sum too old
2023-05-27 11:02:41,868 [DEBUG] sarracenia.moth.amqp putNewMessage published body: {"pubTime": "20230527T150241.865911961", "relPath": "sample.txt", "baseUrl": "http://host", "mode": "664", "size": 335, "mtime": "20230527T150237.927556038", "atime": "20230525T031938.0635721684", "identity": {"method": "sha512", "value": "w5ZwUT1IMAjnQT6TLR9NSLzG5RKijhxq46FjMx5UWtsHM/FNOaYNRmGwonIPfnhE5xUORf3z5dRyI6zdL6ygNw=="}} headers: {} to xpublic under: v03
2023-05-27 11:02:41,868 [DEBUG] amqp collect Closed channel #1

{'_format': 'v03', '_deleteOnPost': {'local_offset', 'new_baseUrl', 'new_dir', 'new_subtopic', 'new_file', 'subtopic', '_format', 'exchange', 'new_relPath', 'post_format'}, 'exchange': 'xpublic', 'local_offset': 0, 'pubTime': '20230527T150241.865911961', 'new_dir': '/tmp', 'new_file': 'sample.txt', 'post_format': 'v03', 'new_baseUrl': 'http://host', 'new_relPath': 'sample.txt', 'new_subtopic': [], 'relPath': 'sample.txt', 'subtopic': [], 'baseUrl': 'http://host', 'mode': '664', 'size': 335, 'mtime': '20230527T150237.927556038', 'atime': '20230525T031938.0635721684', 'identity': {'method': 'sha512', 'value': 'w5ZwUT1IMAjnQT6TLR9NSLzG5RKijhxq46FjMx5UWtsHM/FNOaYNRmGwonIPfnhE5xUORf3z5dRyI6zdL6ygNw=='}}

One can post as many messages as needed with putNewMessage, when finished with posting files, to shut down the connection with the broker cleanly, please close the posting_engine.