{"_id":"57fcc4870312b20e00ac64f9","category":{"_id":"57fcc4860312b20e00ac64c9","__v":0,"version":"57fcc4860312b20e00ac64c0","project":"5435687035740020002a1c04","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-05-01T21:33:28.863Z","from_sync":false,"order":8,"slug":"integration-examples","title":"Integration Examples"},"parentDoc":null,"project":"5435687035740020002a1c04","user":"5435682035740020002a1c01","__v":1,"version":{"_id":"57fcc4860312b20e00ac64c0","project":"5435687035740020002a1c04","__v":1,"createdAt":"2016-10-11T10:52:54.637Z","releaseDate":"2016-10-11T10:52:54.637Z","categories":["57fcc4860312b20e00ac64c1","57fcc4860312b20e00ac64c2","57fcc4860312b20e00ac64c3","57fcc4860312b20e00ac64c4","57fcc4860312b20e00ac64c5","57fcc4860312b20e00ac64c6","57fcc4860312b20e00ac64c7","57fcc4860312b20e00ac64c8","57fcc4860312b20e00ac64c9","57fcc4860312b20e00ac64ca","57fcc4860312b20e00ac64cb"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"[APP-1265], [APP-1035]","version_clean":"5.2.0","version":"5.2"},"updates":[],"next":{"pages":[],"description":""},"createdAt":"2015-05-01T21:34:13.904Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":2,"body":"[block:callout]\n{\n  \"type\": \"info\",\n  \"title\": \"Source Code Available\",\n  \"body\": \"The source code for this example is available on the [Telemetry Agents Examples ](https://github.com/telemetryapp/agent_examples/tree/master/logfile) Github repository.\"\n}\n[/block]\n\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/8KZ7CAOTwumhronqbV2P_methods.png\",\n        \"methods.png\",\n        \"1122\",\n        \"538\",\n        \"#698647\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\n# Requirements\n\nYou'll need an OSX or Linux computer system to run through this guide. Windows is supported with Telemetry but some of the details here would be slightly different and thus it's out of scope.\n\n\n# Installation\n\nYou'll need to install two pieces of software,  our Telemetry Agent, and Sysstatsd.  We created Logwatchd as a sample project that can tail log files and generate simple Graphite data for your system you install it on.  You can find details on how to install these two packages in their respective Github pages:\n\n- [Telemetry Agent](https://github.com/telemetryapp/gotelemetry_agent/releases)\n- [Logwatchd](https://github.com/telemetryapp/logwatchd)\n\n# Configure Board\n\nVisit the [Telemetry Manager](https://manager.telemetryapp.com) application, login and either create a new board or use an existing one. Add a gauge to the board and take note of the *flow_tag*. You'll use this tag to send data to.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/1a743fa-Screen_Shot_2017-07-26_at_10.42.52_AM.png\",\n        \"Screen Shot 2017-07-26 at 10.42.52 AM.png\",\n        263,\n        170,\n        \"#232a34\"\n      ]\n    }\n  ]\n}\n[/block]\n# Configure Agent\n\nNext, we'll need to configure the agent to run the node script and send it to Telemetry.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"#\\n# Server Configuration Parameters\\n#\\n\\n[server]\\n# API token found in the Agents tab in manager.telemetryapp.com\\napi_token = \\\"abc123abc123abc123abc123abc123abc123\\\"\\n# Submit new metrics in a batch at most every 1 second\\nsubmission_interval = \\\"1s\\\"\\n\\n#\\n# Local data storage for aggregation and graphite data\\n#\\n\\n[data]\\n# Location to store\\n# path = \\\"c:\\\\\\\\telemetry_agent.db\\\"\\n# WINDOWS USERS - Uncomment the line above and comment or remove the line below\\npath = \\\"/tmp/telemetry_agent.db\\\"\\n# Expire data after 7 days\\nttl = \\\"7d\\\"\\n\\n#\\n# Graphite Configuration Parameters\\n#\\n\\n[graphite]\\n# Listen on UDP port 2003 on all interfaces for Graphite data\\nlisten_udp = \\\":2003\\\"\\n\\n#\\n# Flow Configurations \\n#\\n\\n#\\n# A specific flow\\n#\\n[[flow]]\\n# The flow tag\\ntag = \\\"graphite_gauge_1\\\"\\n# run the script every 2 seconds\\ninterval = \\\"2s\\\"\\n# The Lua script to run to generate updates\\nscript = \\\"graphite.lua\\\"\",\n      \"language\": \"toml\"\n    }\n  ]\n}\n[/block]\n# Start Logwatchd\n\nFirst, you'll need a log file that you want to monitor on your system.  In the example below I'm using OSX's system.log but you can pick any log file that has relatively frequent updates.   Next you'll need to think of a regex that you want to search for (or use * to match all lines) in order to generate the count of matches.  \n\nLogwatchd will tail the log file and report every interval (10) seconds about how many matching log events were found.\n\nCreate a logwatchd config file that looks like the following:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"# Submit metrics every 10 seconds \\ninterval = 10\\n\\n# The Graphite server (agent) port\\nserver = 'localhost:2003'\\n\\n# A specific search parameter\\n[[search]]\\n# The logfile to search\\nfilename = '/var/log/system.log'\\n# The label for the search\\nlabel = 'launchd'\\n# A regex to search for\\nregex = 'launchd'\",\n      \"language\": \"toml\"\n    }\n  ]\n}\n[/block]\nStart up logwawtchd:\n\n    logwatchd -c logwatchd_config.toml\n    \n\n# Handle Metrics\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"local st = require(\\\"telemetry/storage\\\")\\n\\n-- Replace HOSTNAME with your local systems hostname\\nlocal matchCount = st.series(\\\"HOSTNAME.launchd\\\").last()\\n\\noutput.number = {value = matchCount}\",\n      \"language\": \"lua\"\n    }\n  ]\n}\n[/block]","excerpt":"","slug":"tailing-logfiles","type":"basic","title":"Logfile"}
[block:callout] { "type": "info", "title": "Source Code Available", "body": "The source code for this example is available on the [Telemetry Agents Examples ](https://github.com/telemetryapp/agent_examples/tree/master/logfile) Github repository." } [/block] [block:image] { "images": [ { "image": [ "https://files.readme.io/8KZ7CAOTwumhronqbV2P_methods.png", "methods.png", "1122", "538", "#698647", "" ] } ] } [/block] # Requirements You'll need an OSX or Linux computer system to run through this guide. Windows is supported with Telemetry but some of the details here would be slightly different and thus it's out of scope. # Installation You'll need to install two pieces of software, our Telemetry Agent, and Sysstatsd. We created Logwatchd as a sample project that can tail log files and generate simple Graphite data for your system you install it on. You can find details on how to install these two packages in their respective Github pages: - [Telemetry Agent](https://github.com/telemetryapp/gotelemetry_agent/releases) - [Logwatchd](https://github.com/telemetryapp/logwatchd) # Configure Board Visit the [Telemetry Manager](https://manager.telemetryapp.com) application, login and either create a new board or use an existing one. Add a gauge to the board and take note of the *flow_tag*. You'll use this tag to send data to. [block:image] { "images": [ { "image": [ "https://files.readme.io/1a743fa-Screen_Shot_2017-07-26_at_10.42.52_AM.png", "Screen Shot 2017-07-26 at 10.42.52 AM.png", 263, 170, "#232a34" ] } ] } [/block] # Configure Agent Next, we'll need to configure the agent to run the node script and send it to Telemetry. [block:code] { "codes": [ { "code": "#\n# Server Configuration Parameters\n#\n\n[server]\n# API token found in the Agents tab in manager.telemetryapp.com\napi_token = \"abc123abc123abc123abc123abc123abc123\"\n# Submit new metrics in a batch at most every 1 second\nsubmission_interval = \"1s\"\n\n#\n# Local data storage for aggregation and graphite data\n#\n\n[data]\n# Location to store\n# path = \"c:\\\\telemetry_agent.db\"\n# WINDOWS USERS - Uncomment the line above and comment or remove the line below\npath = \"/tmp/telemetry_agent.db\"\n# Expire data after 7 days\nttl = \"7d\"\n\n#\n# Graphite Configuration Parameters\n#\n\n[graphite]\n# Listen on UDP port 2003 on all interfaces for Graphite data\nlisten_udp = \":2003\"\n\n#\n# Flow Configurations \n#\n\n#\n# A specific flow\n#\n[[flow]]\n# The flow tag\ntag = \"graphite_gauge_1\"\n# run the script every 2 seconds\ninterval = \"2s\"\n# The Lua script to run to generate updates\nscript = \"graphite.lua\"", "language": "toml" } ] } [/block] # Start Logwatchd First, you'll need a log file that you want to monitor on your system. In the example below I'm using OSX's system.log but you can pick any log file that has relatively frequent updates. Next you'll need to think of a regex that you want to search for (or use * to match all lines) in order to generate the count of matches. Logwatchd will tail the log file and report every interval (10) seconds about how many matching log events were found. Create a logwatchd config file that looks like the following: [block:code] { "codes": [ { "code": "# Submit metrics every 10 seconds \ninterval = 10\n\n# The Graphite server (agent) port\nserver = 'localhost:2003'\n\n# A specific search parameter\n[[search]]\n# The logfile to search\nfilename = '/var/log/system.log'\n# The label for the search\nlabel = 'launchd'\n# A regex to search for\nregex = 'launchd'", "language": "toml" } ] } [/block] Start up logwawtchd: logwatchd -c logwatchd_config.toml # Handle Metrics [block:code] { "codes": [ { "code": "local st = require(\"telemetry/storage\")\n\n-- Replace HOSTNAME with your local systems hostname\nlocal matchCount = st.series(\"HOSTNAME.launchd\").last()\n\noutput.number = {value = matchCount}", "language": "lua" } ] } [/block]