Logging

From aptrust
Jump to: navigation, search
PharosPharos is APTrusts web interface to manage deposits and inspect deposit outcomesETag The entity tag is a hash of the object. The ETag reflects changes only to the contents of an object, not its metadata. Application Server Logs

The Ruby application server logs activity in its application logs. Most errors are presented to the user on the web frontend itself, some are only logged in the application logs that are not accessible by the end-user. APTrust staff is spot checking server logs occasionally and errors are reported by Logwatch per email.

Nginx Webserver Logs

The web server logs access to PharosPharos is APTrusts web interface to manage deposits and inspect deposit outcomesETag The entity tag is a hash of the object. The ETag reflects changes only to the contents of an object, not its metadata. and errors from the Ruby application server. These are parsed and evaluated by Logwatch as well and APTrust staff is notified per email if non-regular activities occur.

Exchange - Ingest/Restore/Fixity Services logging
APTrust keeps extensive logs from each micro service from the Exchange suite. Below an example excerpt from downloading an object from ingest in order to be processed on the server. Each result dictionary tracks data and timestamps about the step in the ingest process.
 1 -------- BEGIN aptrust.receiving.templateuniversity.edu/templateuniversity.edu.10822_1046084.tar | Etag: fef0a540264e38912e9f6c4bc3268d9e | Time: 2018-04-09T16:16:07Z --------
 2  {
 3   "WorkItemId": 181485,
 4   "S3Bucket": "aptrust.receiving.templateuniversity.edu",
 5   "S3Key": "templateuniversity.edu.10822_1046084.tar",
 6   "ETag": "fef0a540264e38912e9f6c4bc3268d9e",
 7   "BagPath": "/mnt/lvm/apt/data/templateuniversity.edu/templateuniversity.edu.10822_1046084.tar",
 8   "DBPath": "/mnt/lvm/apt/data/templateuniversity.edu/templateuniversity.edu.10822_1046084.valdb",
 9   "FetchResult": {
10     "Attempted": true,
11     "AttemptNumber": 1,
12     "ErrorIsFatal": false,
13     "Errors": [],
14     "StartedAt": "2018-04-09T16:16:05.215809131Z",
15     "FinishedAt": "2018-04-09T16:16:06.6678436Z",
16     "Retry": true
17   },
18   "UntarResult": {
19     "Attempted": false,
20     "AttemptNumber": 0,
21     "ErrorIsFatal": false,
22     "Errors": [],
23     "StartedAt": "0001-01-01T00:00:00Z",
24     "FinishedAt": "0001-01-01T00:00:00Z",
25     "Retry": true
26   },
27   "ValidateResult": {
28     "Attempted": true,
29     "AttemptNumber": 1,
30     "ErrorIsFatal": false,
31     "Errors": [],
32     "StartedAt": "2018-04-09T16:16:06.906390067Z",
33     "FinishedAt": "2018-04-09T16:16:07.442069031Z",
34     "Retry": true
35   },
36   "StoreResult": {
37     "Attempted": false,
38     "AttemptNumber": 0,
39     "ErrorIsFatal": false,
40     "Errors": [],
41     "StartedAt": "0001-01-01T00:00:00Z",
42     "FinishedAt": "0001-01-01T00:00:00Z",
43     "Retry": true
44   },
45   "RecordResult": {
46     "Attempted": false,
47     "AttemptNumber": 0,
48     "ErrorIsFatal": false,
49     "Errors": [],
50     "StartedAt": "0001-01-01T00:00:00Z",
51     "FinishedAt": "0001-01-01T00:00:00Z",
52     "Retry": true
53   },
54   "CleanupResult": {
55     "Attempted": false,
56     "AttemptNumber": 0,
57     "ErrorIsFatal": false,
58     "Errors": [],
59     "StartedAt": "0001-01-01T00:00:00Z",
60     "FinishedAt": "0001-01-01T00:00:00Z",
61     "Retry": true
62   },
63   "Object": {
64     "state": "A",
65     "created_at": "0001-01-01T00:00:00Z",
66     "updated_at": "0001-01-01T00:00:00Z",
67     "ingest_deleted_from_receiving_at": "0001-01-01T00:00:00Z"
68   }
69 }
70  -------- END aptrust.receiving.templateuniversity.edu/templateuniversity.edu.10822_1046084.tar | Etag: fef0a540264e38912e9f6c4bc3268d9e | Time: 2018-04-09T16:16:07Z --------
Preservation Storage Logging

Activity on the preservation bucket is logged using AWS standard logging to a bucket named `aptrust.preservation.logging` for deeper auditing purposes and security.  This is in addition to any logging already provided by locally coded content services.

AWS Cloudtrail Logs
AWS CloudTrail is a service that enables governance, compliance, operational auditing, and risk auditing of your AWS account. With CloudTrail, you can log, continuously monitor, and retain account activity related to actions across your AWS infrastructure. CloudTrail provides event history of your AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services. This event history simplifies security analysis, resource change tracking, and troubleshooting.[1]
APTrust keeps and "AuditTrail" log that stores activity in all regions, all management activities and all activities on all S3 buckets. The log is continuously stored in an S3 bucket "cloudtrail-logs" and is used as a complete audit trail of all AWS related activities. The trail is currently not processed or triggers any alarms.