Communities
A collaborative platform to connect and grow with like-minded Informaticans across the globe
Product Communities
Connect and collaborate with Informatica experts and champions
Discussions
Have a question? Start a Discussion and get immediate answers you are looking for
User Groups
Customer-organized groups that meet online and in-person. Join today to network, share ideas, and get tips on how to get the most out of Informatica
Get Started
Community Guidelines
Knowledge Center
Troubleshooting documents, product guides, how to videos, best practices, and more
Knowledge Base
One-stop self-service portal for solutions, FAQs, Whitepapers, How Tos, Videos, and more
Support TV
Video channel for step-by-step instructions to use our products, best practices, troubleshooting tips, and much more
Documentation
Information library of the latest product documents
Velocity (Best Practices)
Best practices and use cases from the Implementation team
Learn
Rich resources to help you leverage full capabilities of our products
Trainings
Role-based training programs for the best ROI
Certifications
Get certified on Informatica products. Free, Foundation, or Professional
Product Learning Paths
Free and unlimited modules based on your expertise level and journey
Resources
Library of content to help you leverage the best of Informatica products
Tech Tuesdays Webinars
Most popular webinars on product architecture, best practices, and more
Product Availability Matrix
Product Availability Matrix statements of Informatica products
SupportFlash
Monthly support newsletter
Support Documents
Informatica Support Guide and Statements, Quick Start Guides, and Cloud Product Description Schedule
Product Lifecycle
End of Life statements of Informatica products
Ideas
Events
Change Request Tracking
Marketplace
English
  • Edge Data Streaming 2.4.0
  • All Products

Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Edge Data Streaming
  3. Licenses
  4. Using Informatica Administrator
  5. Creating and Managing the Edge Data Streaming Service
  6. Edge Data Streaming Entity Types
  7. Edge Data Streaming Nodes
  8. Data Connections
  9. Working With Data Flows
  10. Managing the Edge Data Streaming Components
  11. Security
  12. High Availability
  13. Disaster Recovery
  14. Monitoring Edge Data Streaming Entities
  15. Appendix A: Troubleshooting
  16. Appendix B: Frequently Asked Questions
  17. Appendix C: Regular Expressions
  18. Appendix D: Command Line Program
  19. Appendix E: Configuring Edge Data Streaming to Work With a ZooKeeper Observer
  20. Appendix F: Glossary

User Guide

User Guide

Back Next

Pattern Examples

Pattern Examples

The following examples describe some patterns that you can configure:

Pattern Without Custom Regular Expression

If you want to transform the input
vds
 into JSON format, configure the following properties:
  • Entity Name. Enter a name for the transformation.
  • Pattern. Enter
    %{WORD:myword}
When you deploy the data flow, you get the following output:
{"myword": [["vds"]]}

Pattern With Custom Regular Expression

If you want to transform the input
ABC
 into JSON format, configure the following properties:
  • Entity Name. Enter a name for the transformation.
  • Pattern. Enter
    %{ALLCAPS:capword}
  • Custom Regex. Enter
    ALLCAPS=[A-Z]+
When you deploy the data flow, you get the following output:
{"capword":"ABC"}

Pattern to Convert a Log Line

If you want to transform the log line
Jul 9 22:41:51 myserver sshd[4295]: Failed password for invalid user myuser from 220.113.135.154 port 55993 ssh2
 into JSON format, configure the following properties:
  • Entity Name. Enter a name for the transformation.
  • Pattern. Enter
    %{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host_target} sshd\[%{BASE10NUM}\]: Failed password for (invalid user |)%{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2
When you deploy the data flow, you get the following output:
{"BASE10NUM":4295,"HOUR":22,"MINUTE":41,"MONTH":"Jul","MONTHDAY":9,"SECOND":51,"TIME":"22:41:51","host_target":"myserver","port":55993,"src_ip":"220.113.135.154","timestamp":"Jul 9 22:41:51","username":"myuser"}

Pattern to Convert a Syslog Log Line

If you want to transform the log line
Oct 17 08:59:00 suod newsyslog[6215]: logfile turned over
 into JSON format, configure the following properties:
  • Entity Name. Enter a name for the transformation.
  • Pattern. Enter
    %{SYSLOGBASE} %{GREEDYDATA}
When you deploy the data flow, you get the following output:
{"SYSLOGBASE":[["Oct 17 08:59:00 suod newsyslog[6215]:"]],"timestamp":[["Oct 17 08:59:00"]],"MONTH":[["Oct"]],"MONTHDAY":[["17"]],"TIME":[["08:59:00"]],"HOUR":[["08"]],"MINUTE":[["59"]],"SECOND":[["00"]],"SYSLOGFACILITY":[[null]],"facility":[[null]],"priority":[[null]],"logsource":[["suod"]],"IPORHOST":[["suod"]],"HOSTNAME":[["suod"]],"IP":[[null]],"IPV6":[[null]],"IPV4":[[null]],"SYSLOGPROG":[["newsyslog[6215]"]],"program":[["newsyslog"]],"pid":[["6215"]],"GREEDYDATA":[["logfile turned over"]]}

Pattern to Convert an Apache Log Line

If you want to transform the log line
127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326
 into JSON format, configure the following properties:
  • Entity Name. Enter a name for the transformation.
  • Pattern. Enter
    %{COMMONAPACHELOG}
When you deploy the data flow, you get the following output:
{"COMMONAPACHELOG":[["127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET/apache_pb.gifHTTP/1.0" 200 2326"]],"clientip":[["127.0.0.1"]],"HOSTNAME":[["127.0.0.1"]],"IP":[[null]],"IPV6":[[null]],"IPV4":[[null]],"ident":[["-"]],"USERNAME":[["-","frank"]],"auth":[["frank"]],"timestamp":[["10/Oct/2000:13:55:36 -0700"]],"MONTHDAY":[["10"]],"MONTH":[["Oct"]],"YEAR":[["2000"]],"TIME":[["13:55:36"]],"HOUR":[["13"]],"MINUTE":[["55"]],"SECOND":[["36"]],"INT":[["-0700"]],"verb":[["GET"]],"request":[["/apache_pb.gif"]],"httpversion":[["1.0"]],"BASE10NUM":[["1.0","200","2326"]],"rawrequest":[[null]],"response":[["200"]],"bytes":[["2326"]]}

Pattern to Convert a log4j Log Line

If you want to transform the log line
2015-03-03 11:35:53,759 [WARN ] [lbm:Thread-27] Core-5688-1883: timer returned error 5 [CoreApi-5688-3337: lbm_socket_sendb send/sendto: (10049) The requested address is not valid in its context.
 into JSON format, configure the following properties:
  • Entity Name. Enter a name for the transformation.
  • Pattern. Enter
    %{TIMESTAMP_ISO8601} \[%{LOGLEVEL} %{GREEDYDATA}
When you deploy the data flow, you get the following output:
{"TIMESTAMP_ISO8601":[["2015-03-03 11:35:53,759"]],"YEAR":[["2015"]],"MONTHNUM":[["03"]],"MONTHDAY":[["03"]],"HOUR":[["11",null]],"MINUTE":[["35",null]],"SECOND":[["53,759"]],"ISO8601_TIMEZONE":[[null]],"LOGLEVEL":[["WARN"]],"GREEDYDATA":[["] [lbm:Thread-27] Core-5688-1883: timer returned error 5 [CoreApi-5688-3337: lbm_socket_sendb send/sendto: (10049) The requested address is not valid in its context."]]}
0 COMMENTS
We’d like to hear from you!