Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 'logging-api' branch to master. #1685

Merged
merged 112 commits into from
Mar 31, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
f2becb0
Logging API usage docs.
tseaver Mar 2, 2016
57e8424
Add logging connection and client.
tseaver Mar 2, 2016
139519a
Clean up copy-pasta:
tseaver Mar 2, 2016
08d9565
Drop redundant method overrides, left over from Pub/Sub.
tseaver Mar 2, 2016
586113f
Add assertions to placeholder testcase methods.
tseaver Mar 2, 2016
44f9558
Fix docs coverage for logging.
tseaver Mar 3, 2016
b5798fd
Add 'Logger' class and factory.
tseaver Mar 3, 2016
d38fd23
Fix copyright year.
tseaver Mar 3, 2016
95238be
Merge pull request #1583 from tseaver/pylint-ignore_six_and_google_pr…
tseaver Mar 8, 2016
b466893
Add 'Logger.log_text' method.
tseaver Mar 3, 2016
cb3772d
Fix copy-pasta in 'Client.logger' docstring.
tseaver Mar 8, 2016
b04eaa6
Merge pull request #1567 from tseaver/logging-logger_log_text
tseaver Mar 9, 2016
9ec6add
Add 'Logger.log_struct' method.
tseaver Mar 3, 2016
2b506b9
Merge branch 'master' into logging-api
tseaver Mar 9, 2016
816d1eb
Merge pull request #1568 from tseaver/logging-logger_log_struct
tseaver Mar 9, 2016
b0ad3e0
Add 'Logger.delete' method.
tseaver Mar 7, 2016
5c2f2ed
Merge pull request #1581 from tseaver/logging-logger_delete
tseaver Mar 9, 2016
e898bca
Merge branch 'master' into logging-api
tseaver Mar 9, 2016
d7a2e72
Add 'Client.list_entries'.
tseaver Mar 3, 2016
68baa1b
Merge pull request #1569 from tseaver/logging-client_list_entries
tseaver Mar 11, 2016
e4b4b0d
Add 'Logger.list_entries'.
tseaver Mar 9, 2016
b8151b1
Merge pull request #1574 from tseaver/logging-logger_list_entries
tseaver Mar 11, 2016
a9c8515
Add 'Sink.create' API wrapper and 'Client.sink' factory.
tseaver Mar 10, 2016
6f3d1a7
Merge pull request #1596 from tseaver/logging-sink_create
tseaver Mar 11, 2016
47cf5ab
Add 'Sink.exists' API wrapper.
tseaver Mar 10, 2016
2c94675
Merge pull request #1597 from tseaver/logging-sink_exists
tseaver Mar 11, 2016
629418b
Add 'Sink.reload' API wrapper.
tseaver Mar 10, 2016
014b619
Add 'metric.Metric' class and its 'create' API wrapper.
tseaver Mar 7, 2016
427f96d
Merge pull request #1598 from tseaver/logging-sink_reload
tseaver Mar 11, 2016
dd1d44a
Add 'Sink.update' API wrapper.
tseaver Mar 10, 2016
cd41be4
Merge pull request #1589 from tseaver/logging-client_logger_docstring…
tseaver Mar 11, 2016
b51de13
Merge pull request #1599 from tseaver/logging-sink_update
tseaver Mar 12, 2016
6407219
Add 'Sink.delete' API wrapper.
tseaver Mar 10, 2016
6e6a9c4
Merge pull request #1600 from tseaver/logging-sink_delete
tseaver Mar 12, 2016
81cd850
Add 'Sink.from_api_repr'.
tseaver Mar 11, 2016
b63d62d
Add 'Client.list_sinks' API wrapper.
tseaver Mar 11, 2016
5baba70
Use tuple for format string arg.
tseaver Mar 12, 2016
a8cdf27
Merge pull request #1601 from tseaver/logging-client_list_sinks
tseaver Mar 12, 2016
f2d8ad6
Merge pull request #1584 from tseaver/logging-metric_create
tseaver Mar 12, 2016
9a56835
Add 'Metrics.exists' API wrapper.
tseaver Mar 8, 2016
e16b542
Merge pull request #1585 from tseaver/logging-metric_exists
tseaver Mar 12, 2016
b66acf5
Add 'Metrics.reload' API wrapper.
tseaver Mar 8, 2016
245e72e
Merge pull request #1586 from tseaver/logging-metric_reload
tseaver Mar 13, 2016
3010551
Add 'Metrics.update' API wrapper.
tseaver Mar 8, 2016
23feb30
Restore doc assertion that metric exists.
tseaver Mar 14, 2016
0283a36
Merge pull request #1587 from tseaver/logging-metric_update
tseaver Mar 14, 2016
75b1e79
Add 'Metrics.delete' API wrapper.
tseaver Mar 8, 2016
92a7bb4
Merge pull request #1588 from tseaver/logging-metric_delete
tseaver Mar 14, 2016
04be546
Add 'Metric.from_api_repr'.
tseaver Mar 8, 2016
4298806
Add 'Client.list_metrics' API wrapper.
tseaver Mar 8, 2016
5d342fa
Add 'Client.metric' factory.
tseaver Mar 8, 2016
affbc5d
Undo extra linewrap, maybe added during conflict resolution.
tseaver Mar 14, 2016
3101fb0
Drop no-longer-needed pylint disable.
tseaver Mar 14, 2016
02eecd1
Fix docstring copy-pasta.
tseaver Mar 14, 2016
f1ca149
Re-add docstring line lost in conflict resolution.
tseaver Mar 14, 2016
5648192
Merge pull request #1590 from tseaver/logging-client_list_metrics
tseaver Mar 14, 2016
15526df
Fix verb/target for 'Metric.create'/'Sink.create'.
tseaver Mar 15, 2016
466409e
Merge pull request #1616 from tseaver/logging-sink_metric_fix_create_…
tseaver Mar 17, 2016
d7062b0
System test for 'logger.log_text' and 'logger.log_struct'.
tseaver Mar 15, 2016
57fed8a
Merge pull request #1609 from tseaver/logging-system_test-logger_log
tseaver Mar 17, 2016
97be817
Add system test for 'Client.metric'/'Metric.create'.
tseaver Mar 15, 2016
93443d1
Merge pull request #1610 from tseaver/logging-system_test-metric_create
tseaver Mar 18, 2016
61004b7
Add system test for 'Client.list_metrics'.
tseaver Mar 15, 2016
4d21791
Merge pull request #1611 from tseaver/logging-system_test-client_list…
tseaver Mar 18, 2016
e0492bd
Add system test for 'Metric.reload'.
tseaver Mar 15, 2016
4b88a63
Merge pull request #1612 from tseaver/logging-system_test-metric_reload
tseaver Mar 18, 2016
e3d1783
Add system test for 'Metric.update'.
tseaver Mar 15, 2016
303f2a8
Merge pull request #1613 from tseaver/logging-system_test-metric_update
tseaver Mar 18, 2016
76a47d9
Add system test for 'Sink.create'.
tseaver Mar 15, 2016
7c52009
Add support for parsing log entries w/ 'protoPayload' from resources.
tseaver Mar 7, 2016
fd8965c
Test '_BaseEntry' directly, drop tests for methodless subclasses.
tseaver Mar 22, 2016
9f37a05
Merge pull request #1578 from tseaver/logging-parse_protobuf_entries
tseaver Mar 22, 2016
0adbd70
Merge pull request #1628 from tseaver/logging-system_test-sink_create
tseaver Mar 22, 2016
cf7c371
Merge branch 'master' into logging-api
tseaver Mar 23, 2016
5d48c65
Add 'Logger.log_proto'.
tseaver Mar 25, 2016
f6e9de8
Add 'ProtobufEntry.parse_message' helper.
tseaver Mar 25, 2016
1f1ecad
Add system test for 'Sink.create' using a Bigquery dataset.
tseaver Mar 24, 2016
2de3e0a
Drop use of 'google.protobuf.unittest_pb2.BoolMessage'.
tseaver Mar 25, 2016
e9a01ea
Document 'ProtobufEntry.parse_message' mutates passed-in message in …
tseaver Mar 26, 2016
cd18373
Merge pull request #1661 from tseaver/1577-logging-protobuf_load_dump
tseaver Mar 26, 2016
d93f807
Merge branch 'master' into logging-api
tseaver Mar 28, 2016
159d8a3
Add 'logger.Batch' for logging multiple entries via a single API call.
tseaver Mar 28, 2016
145ec60
Move repeated 'logName'/'resource' element to wrapper.
tseaver Mar 28, 2016
9a9a376
Add coverage for can't-get-here 'else' clause.
tseaver Mar 28, 2016
44d8e86
Clean up docstring copy-pasta.
tseaver Mar 28, 2016
8666e63
Merge pull request #1666 from tseaver/logging-cleanup_docstring_copy_…
tseaver Mar 28, 2016
1b822fe
Merge pull request #1658 from tseaver/logging-system_tests-sink_creat…
tseaver Mar 28, 2016
fad7f4c
Merge pull request #1665 from tseaver/logging-multi_entry_batch
tseaver Mar 28, 2016
dfd815a
Track labels set on an entry by the backend.
tseaver Mar 28, 2016
2c4c5b3
Allow setting default labels on a logger instance.
tseaver Mar 28, 2016
64e2f1c
Add support for logging entries with labels.
tseaver Mar 28, 2016
53df5fc
Add 'Logger.path' property.
tseaver Mar 28, 2016
7aa9629
Add label support to the newly-added batch feature.
tseaver Mar 28, 2016
43a0936
Fix incorrect 'structPayload' entry in 'Batch.commit'.
tseaver Mar 28, 2016
5cbeaca
Factor out construction of log entry resource into a helper.
tseaver Mar 28, 2016
f1b9261
Merge pull request #1668 from tseaver/logging-support_entry_labels
tseaver Mar 28, 2016
36a9520
Add 'severity'/'http_request' attrs to '_BaseEntry'.
tseaver Mar 28, 2016
145cbfb
Add support for logging w/ per-request metadata.
tseaver Mar 28, 2016
79c8e2c
Merge pull request #1669 from tseaver/logging-support_entry_insertid_…
tseaver Mar 29, 2016
660441d
Avoid reusing logger name between tests.
tseaver Mar 30, 2016
a7749e9
Add system test for 'Sink.reload'.
tseaver Mar 30, 2016
81be985
Merge branch 'master' into logging-api
tseaver Mar 30, 2016
9a7df6f
Merge pull request #1677 from tseaver/logging-system_tests-avoid_reus…
tseaver Mar 30, 2016
ae88612
Merge pull request #1678 from tseaver/logging-system_tests-sink_reload
tseaver Mar 30, 2016
1800393
Add system test for sink creation w/ pubsub topic.
tseaver Mar 23, 2016
a0c2d89
Add system test for 'Sink.update'.
tseaver Mar 30, 2016
95f6a67
Add system tests for logging w/ insert_id/severty/http_request metdata.
tseaver Mar 30, 2016
9bd2880
Add exponential backoff for deletion failures.
tseaver Mar 30, 2016
7ae4fd5
Merge pull request #1681 from tseaver/logging-system_tests-sink_update
tseaver Mar 30, 2016
451fec9
Merge pull request #1682 from tseaver/logging-system_tests-log_w_meta…
tseaver Mar 30, 2016
c3d0977
Merge pull request #1683 from tseaver/logging-system_tests-retry_dele…
tseaver Mar 30, 2016
b16e272
Merge pull request #1680 from tseaver/logging-system_tests-sink_creat…
tseaver Mar 31, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,18 @@
search-index
search-document

.. toctree::
:maxdepth: 0
:hidden:
:caption: Cloud Logging

logging-usage
Client <logging-client>
logging-logger
logging-entries
logging-metric
logging-sink

.. toctree::
:maxdepth: 0
:hidden:
Expand Down
16 changes: 16 additions & 0 deletions docs/logging-client.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Logging Client
==============

.. automodule:: gcloud.logging.client
:members:
:undoc-members:
:show-inheritance:

Connection
~~~~~~~~~~

.. automodule:: gcloud.logging.connection
:members:
:undoc-members:
:show-inheritance:

8 changes: 8 additions & 0 deletions docs/logging-entries.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Entries
=======

.. automodule:: gcloud.logging.entries
:members:
:undoc-members:
:show-inheritance:

8 changes: 8 additions & 0 deletions docs/logging-logger.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Logger
======

.. automodule:: gcloud.logging.logger
:members:
:undoc-members:
:show-inheritance:

7 changes: 7 additions & 0 deletions docs/logging-metric.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Metrics
=======

.. automodule:: gcloud.logging.metric
:members:
:undoc-members:
:show-inheritance:
7 changes: 7 additions & 0 deletions docs/logging-sink.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Sinks
=====

.. automodule:: gcloud.logging.sink
:members:
:undoc-members:
:show-inheritance:
315 changes: 315 additions & 0 deletions docs/logging-usage.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,315 @@
Using the API
=============


Authentication and Configuration
--------------------------------

- For an overview of authentication in ``gcloud-python``,
see :doc:`gcloud-auth`.

- In addition to any authentication configuration, you should also set the
:envvar:`GCLOUD_PROJECT` environment variable for the project you'd like
to interact with. If you are Google App Engine or Google Compute Engine
this will be detected automatically.

- After configuring your environment, create a
:class:`Client <gcloud.logging.client.Client>`

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()

or pass in ``credentials`` and ``project`` explicitly

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client(project='my-project', credentials=creds)


Writing log entries
-------------------

Write a simple text entry to a logger.

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> logger = client.logger('log_name')
>>> logger.log_text("A simple entry") # API call

Write a dictionary entry to a logger.

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> logger = client.logger('log_name')
>>> logger.log_struct(
... message="My second entry",
... weather="partly cloudy") # API call


Retrieving log entries
----------------------

Fetch entries for the default project.

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> entries, token = client.list_entries() # API call
>>> for entry in entries:
... timestamp = entry.timestamp.isoformat()
... print('%sZ: %s | %s' %
... (timestamp, entry.text_payload, entry.struct_payload))
2016-02-17T20:35:49.031864072Z: A simple entry | None
2016-02-17T20:38:15.944418531Z: None | {'message': 'My second entry', 'weather': 'partly cloudy'}

Fetch entries across multiple projects.

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> entries, token = client.list_entries(
... project_ids=['one-project', 'another-project']) # API call

Filter entries retrieved using the `Advanced Logs Filters`_ syntax

.. _Advanced Logs Filters: https://cloud.google.com/logging/docs/view/advanced_filters

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> FILTER = "log:log_name AND textPayload:simple"
>>> entries, token = client.list_entries(filter=FILTER) # API call

Sort entries in descending timestamp order.

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> entries, token = client.list_entries(order_by=logging.DESCENDING) # API call

Retrieve entries in batches of 10, iterating until done.

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> retrieved = []
>>> token = None
>>> while True:
... entries, token = client.list_entries(page_size=10, page_token=token) # API call
... retrieved.extend(entries)
... if token is None:
... break

Retrieve entries for a single logger, sorting in descending timestamp order:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> logger = client.logger('log_name')
>>> entries, token = logger.list_entries(order_by=logging.DESCENDING) # API call

Delete all entries for a logger
-------------------------------

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> logger = client.logger('log_name')
>>> logger.delete_entries() # API call


Manage log metrics
------------------

Metrics are counters of entries which match a given filter. They can be
used within Cloud Monitoring to create charts and alerts.

Create a metric:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> metric = client.metric(
... "robots", "Robots all up in your server",
... filter='log:apache-access AND textPayload:robot')
>>> metric.exists() # API call
False
>>> metric.create() # API call
>>> metric.exists() # API call
True

List all metrics for a project:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> metrics, token = client.list_metrics()
>>> len(metrics)
1
>>> metric = metrics[0]
>>> metric.name
"robots"

Refresh local information about a metric:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> metric = client.metric("robots")
>>> metric.reload() # API call
>>> metric.description
"Robots all up in your server"
>>> metric.filter
"log:apache-access AND textPayload:robot"

Update a metric:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> metric = client.metric("robots")
>>> metric.exists() # API call
True
>>> metric.reload() # API call
>>> metric.description = "Danger, Will Robinson!"
>>> metric.update() # API call

Delete a metric:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> metric = client.metric("robots")
>>> metric.exists() # API call
True
>>> metric.delete() # API call
>>> metric.exists() # API call
False


Export log entries using sinks
------------------------------

Sinks allow exporting entries which match a given filter to Cloud Storage
buckets, BigQuery datasets, or Cloud Pub/Sub topics.

Create a Cloud Storage sink:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> sink = client.sink(
... "robots-storage",
... filter='log:apache-access AND textPayload:robot')
>>> sink.storage_bucket = "my-bucket-name"
>>> sink.exists() # API call
False
>>> sink.create() # API call
>>> sink.exists() # API call
True

Create a BigQuery sink:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> sink = client.sink(
... "robots-bq",
... filter='log:apache-access AND textPayload:robot')
>>> sink.bigquery_dataset = "projects/my-project/datasets/my-dataset"
>>> sink.exists() # API call
False
>>> sink.create() # API call
>>> sink.exists() # API call
True

Create a Cloud Pub/Sub sink:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> sink = client.sink(
... "robots-pubsub",
... filter='log:apache-access AND textPayload:robot')
>>> sink.pubsub_topic = 'projects/my-project/topics/my-topic'
>>> sink.exists() # API call
False
>>> sink.create() # API call
>>> sink.exists() # API call
True

List all sinks for a project:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> sinks, token = client.list_sinks()
>>> for sink in sinks:
... print('%s: %s' % (sink.name, sink.destination))
robots-storage: storage.googleapis.com/my-bucket-name
robots-bq: bigquery.googleapis.com/projects/my-project/datasets/my-dataset
robots-pubsub: pubsub.googleapis.com/projects/my-project/topics/my-topic

Refresh local information about a sink:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> sink = client.sink('robots-storage')
>>> sink.filter is None
True
>>> sink.reload() # API call
>>> sink.filter
'log:apache-access AND textPayload:robot'
>>> sink.destination
'storage.googleapis.com/my-bucket-name'

Update a sink:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> sink = client.sink("robots")
>>> sink.reload() # API call
>>> sink.filter = "log:apache-access"
>>> sink.update() # API call

Delete a sink:

.. doctest::

>>> from gcloud import logging
>>> client = logging.Client()
>>> sink = client.sink(
... "robots",
... filter='log:apache-access AND textPayload:robot')
>>> sink.exists() # API call
True
>>> sink.delete() # API call
>>> sink.exists() # API call
False
Loading