Category: Messaging Infrastructure

Which of these MQ mistakes are you making?

Posted on | by Jeff Eveland
IBM MQ Logo

IBM MQ Tips Straight From TxMQ Subject Matter Experts

At TxMQ we have extensive experience helping clients with their IBM MQ deployments including planning, integrations, management, upgrades, & enhancements. Throughout the years we’ve seen just about everything, and we have found there are some common mistakes that are easy to stay away from with a little insight. Here are some a few tips to keep you up and running:IBM MQ is a powerful tool that makes a huge difference in your life every day, but for most of us you only notice it when it’s not working. One small mistake can cause havoc to your entire system.

1. Don’t use MQ as a database. MQ is for moving messages from one application or system to another. Databases are the data repository of choice. MQ resources are most optimized when considering data throughput and message delivery efficiency. IBM MQ is optimized when messages are kept small.

2. Don’t expect assured delivery if you didn’t design for it! Assured one-time message delivery is provided by IBM MQ through setting message persistence and advanced planning in the application integration design process. This plans for poisoned message handling that could otherwise cause issues and failures or worse, unexpected results. Know your business requirements for quality of service for message delivery and make sure your integration design accommodates advanced settings and features as appropriate.

3. Don’t give every application their own Queue Manager! Sometimes yes, sometimes no. Understand how to analyze what is best for your needs. Optimize your messaging architecture for shared messaging to control infrastructure costs and optimize operational support resources.

4. Don’t fall out of support. While TxMQ can offer support for OUT of support products, it’s costly to let support lapse on current projects, and even more so if you have to play catch up!

5. Don’t forget monitoring! MQ is powerful and stable, but if a problem occurs, you want to know about it right away. Don’t wait until your XMITQs and application queues fill up and bring the Queue Manager down before you respond!

In the cloud, on mobile, on-prem or in the IoT, IBM MQ simplifies, accelerates, and facilitates security-rich data exchange. Keep your MQ running smoothly, reach out to talk with one of our Subject Matter Experts today!

If you like our content and would like to keep up to date with TxMQ don’t forget to sign up for our monthly newsletter.

 

IBM WebSphere Application Server (WAS) v.7 & v.8, and WebSphere MQ v.7.5 End of Support: April 30, 2018

Posted on | by Jeff Eveland

Are you presently running on WAS versions 7 or 8?
   Are you leveraging WebSphere MQ version 7.5?

Time is running out, IBM WebSphere Application Server (WAS) v.7 & v.8, and WebSphere MQ v.7.5 support ends in less than 6 months. As of April 30th 2018, IBM will discontinue support on all WebSphere Application Server versions 7.0.x & v8.0.x; and WebSphere MQ v7.5.x.

It’s recommended that you migrate to WebSphere Application Server v.9 to avoid potential security issues that may occur on the early, unsupported versions of WAS (and Java).
It’s also recommended that you upgrade to IBM MQ version 9.0.x, to leverage new features, and avoid costly premium support fees from IBM.

Why should you go through an upgrade?

Many security risks can percolate when running back-level software, especially WAS running on older Java versions. If you’re currently running on software versions that are out of support, finding the right support team to put out your unexpected fires can be tricky and might just blow the budget.
Upgrading WAS & MQ to supported versions will allow you to tap into new and expanding capabilities, and updated performance enhancements while also protecting yourself from unnecessary, completely avoidable security risks and added support costs.

WebSphere Application Server v.9 Highlights

WebSphere Application Server v.9.0 offers unparalleled functionality to deliver modern applications and services quickly, securely and efficiently.

When you upgrade to v.9.0, you’ll enjoy several upgrade perks including:
  • Java EE 7 compliant architecture.
  • DevOps workflows.
  • Easy connection between your on-prem apps and IBM Bluemix services (including IBM Watson).
  • Container technology that enables greater development and deployment agility.
  • Deployment on Pivotal Cloud Foundry, Azure, Openshift, Amazon Web Services and Bluemix.
  • Ability to provision workloads to IBM cloud (for VMware customers).
  • Enhancements to WebSphere extreme scale that have improved response times and time-to-configuration.

 

IBM MQ v.9.0.4 Highlights

With the latest update moving to MQ V9.0.4, there are even more substantial updates of useful features for IBM MQ, even beyond what came with versions 8 (z/OS) & 8.5.

When you upgrade to v.9.0.4, enhancements include:
  • Additional commands supported as part of the REST API for admin.
  • Availability of a ‘catch-all’ for MQSC commands as part of the REST API for admin.
  • Ability to use a single MQ V9.0.4 Queue Manager as a single point gateway for REST API based admin of other MQ environments including older MQ versions such as MQ V9 LTS and MQ V8.
  • Ability to use MQ V9.0.4 as a proxy for IBM Cloud Product Insights reporting across older deployed versions of MQ.
  • Availability of an enhanced MQ bridge for Salesforce.
  • Initial availability of a new programmatic REST API for messaging applications.

This upgrade cycle also offers you the opportunity to evaluate the MQ Appliance. Talk to TxMQ to see if the MQ Appliance is a good option for your messaging environment.

What's your WebSphere Migration Plan? Let's talk about it!

Why work with an IBM Business Partner to upgrade your IBM Software?

You can choose to work with IBM directly – we can’t (and won’t) stop you – but your budget just might. Working with a premier IBM business partner allows you to accomplish the same task with the same quality, but at a fraction of the price IBM will charge you, with more personal attention and much speedier response times.
Also, IBM business partners are typically niche players, uniquely qualified to assist in your company’s migration planning and execution. They’ll offer you and your company much more customized and consistent attention. Plus, you’ll probably be working with ex-IBMers anyway, who’ve turned in their blue nametags to find greater opportunities working within the business partner network.

There are plenty of things to consider when migrating your software from outdated versions to more current versions.

TxMQ is a premier IBM business partner that works with customers to oversee and manage software migration and upgrade planning. TxMQ subject matter experts are uniquely positioned with relevant experience, allowing them to help a wide range of customers determine the best solution for their migration needs.
Get in touch with us today to discuss your migration and back-level support options. It’s never too late to begin planning and executing your version upgrades.

To check on your IBM Software lifecycle, simply search your product name and version on this IBM page or, give TxMQ a call…

Why ETL Tools Might Hobble Your Real-Time Reporting & Analytics

Posted on | by admin

Companies report large investments in their data warehousing (DW) or Business Intelligence (BI) systems with a large portion of the software budget spent on extract, transformation and load (ETL) tools that provide ease of populating such warehouses, and the ability to manipulate data to map to the new schemas and data structures.

Companies do need DW or BI for analytics on sales, forecasting, stock management and so much more, and they certainly wouldn’t want to run the additional analytics workloads on top of already taxed core systems, so keeping them isolated is a good idea and a solid architectural decision. Considering, however, the extended infrastructure and support costs involved with managing a new ETL layer, it’s not just the building of them that demands effort. There’s the ongoing scheduling of jobs, on-call support (when jobs fail), the challenge of an ever-shrinking batch window when such jobs are able to run without affecting other production workloads, and other such considerations which make the initial warehouse implementation expense look like penny candy.

So not only do such systems come with extravagant cost, but to make matters worse, the vast majority of jobs run overnight. That means, in most cases, the best possible scenario is that you’re looking at yesterday’s data (not today’s). All your expensive reports and analytics are at least a day later than what you require. What happened to the promised near-realtime information you were expecting?
Contrast the typical BI/DW architecture to the better option of building out your analytics and report processing using realtime routing and transformation with tools such as IBM MQ, DataPower and Integration Bus. Most of the application stacks that process this data in realtime have all the related keys in memory –customer number or ids, order numbers, account details, etc. – and are using them to create or update the core systems. Why duplicate all of that again in your BI/DW ETL layer? If you do, you’re dependent on ETLs going into the core systems to find what happened during that period and extracting all that data again to put it somewhere else.

Alongside this, most organizations are already running application messaging and notifications between applications. lf you have all the data keys in memory, use a DW object, method, function or macro to drop the data as an application message into your messaging layer. The message can then be routed to your DW or BI environment for Transformation and Loading there, no Extraction needed, and you can get rid of your ETL tools.

Simplify your environments and lower the cost of operation. If you have multiple DW or BI environments, use the Pub/Sub capabilities of IBM MQ to distribute the message. You may be exchanging a nominal increase in CPU for eliminating problems, headaches and costs to your DW or BI.

Rethinking your strategy in terms of EAI while removing the whole process and overhead of ETL may indeed bring your whole business analytics to the near-realtime reporting and analytics you expected. Consider that your strategic payoff. Best regards in your architecture endeavors!
Image by Mark Morgan.

Oracle Announces End of Support For JD Edwards EnterpriseONE Technology Foundation

Posted on | by Jeff Eveland

Long title. What’s this about?
In short, back in 2010, Oracle announced the withdrawal of JDE EnterpriseOne Technology Foundation. The final nail in this coffin comes on September 30, 2016, when technical support officially ends.
What this means is that for many customers (and I’ll get into particulars shortly) there’s a requirement to make a critical decision to either move to Oracle’s Red Stack, or procure new IBM licenses in order to remain on IBM’s Blue Stack.
There’s a variety of customers running the stack, and nearly as wide a variety of options for how companies may have deployed their JDE solution. WebSphere with DB2 is the original and most common. WebSphere with Oracle as the backend is another common combo. And there’s a variety of other blends of supported web/application servers, database servers and related middleware.
Regardless of the configuration, in most cases, these products were part of the bundled solution that customers licensed from Oracle, and now a decision point’s been reached.
This doesn’t mean Oracle’s dropping support for IBM products. This does mean there’s a change in the way they’re licensed.
So what is “Technology Foundation”?
To quote from Oracle’s documents verbatim: Technology Foundation is an Oracle product that provides license for the following components:

  • JD Edwards EnterpriseOne Core Tools and Infrastructure, the native design time and runtime tools required to run JD Edwards EnterpriseOne application modules
  • IBM DB2 for Linux, Unix, and Windows, limited to use with JD Edwards EnterpriseOne programs
  • IBM WebSphere Application Server, limited to use with JD Edwards EnterpriseOne programs
  • IBM WebSphere Portal, as contained in JD Edwards EnterpriseOne Collaborative Portal

Technology Foundation is also referred to by the nickname “Blue Stack.”
If your license for JD Edwards EnterpriseOne applications includes an item called “Technology Foundation” or “Technology Foundation Upgrade,” this affects you.
If there are any other terms like “Oracle Technology Foundation,” then this change does NOT affect you. This is also different then the foundation for JD Edwards World.
So what now? In short, if you have Blue Stack, you should contact TxMQ or IBM immediately to acquire your own licensed products to continue to run your Oracle solution. TxMQ can offer aggressive discounts to Oracle customers subject to certain terms and conditions. Contact us for pricing details. We can help with pricing, as well as with any needed migration planning and implementation support.
Contact chuck@txmq.com for immediate price quotes and migration planning today.
Image from Håkan Dahlström.

How Do I Support My Back-Level IBM Software (And Not Blow The Budget)?

Posted on | by Jeff Eveland

So you’re running outdated, obsolete, out-of-support versions of some of your core systems.? WebSphere MQ maybe? or WebSphere Process Server or Datapower…the list is endless.
Staff turnover may be your pain point – a lack of in-house skills – or maybe it’s lack of budget to upgrade to newer, in-support systems. A lost of times it’s just a matter of application dependencies, where you can’t get something to work in QA, and you’re not ready to migrate to current versions just yet.
The problem is that management requires you to be under support. So you get a quote from IBM to support your older software, and the price tag is astronomical – not even in the same solar system as your budget.
The good news is you do have options.

We were able to offer a 6-month support package, which eventually ran 9 months in total. Total cost was under $1,000 a month.

Here at TxMQ, we have a mature and extensive migration practice, but we also offer 24×7 support (available as either 100% onshore, or part onshore, part offshore) for back-level IBM software and product support – all at a fraction of IBM rates.
Our support starts at under $1,000 a month and scales with your size and needs.
TxMQ has been supporting IBM customers for over 35 years. We have teams of architects, programmers, engineers and others across the US and Canada supporting a variety of enterprise needs.
Case Studies
A medical-equipment manufacturer planned to migrate from unsupported versions of MQ and Message Broker. The migration would run 6 to 9 months past end-of-support, but the quote from IBM for premium support was well beyond budget.
The manufacturer reached out to TxMQ and we were able to offer a 6-month support package, which eventually ran 9 months in total. Total cost was under $1,000 a month.
Another customer (a large health-insurance payer) faced a similar situation. This customer was running WebSphere Process Server, Ilog, Process Manager, WAS, MQ, WSRR, Tivoli Monitoring, and outdated DataPower appliances. TxMQ built an executed a comprehensive “safety net” plan to support this customer’s entire stack during a very extensive migration period.
It’s never a good idea to run unsupported software – especially in these times of highly visible compliance and regulatory requirements.
In addition to specific middleware and application support, TxMQ can also work to build a compliance framework to ensure you’re operating within IBM’s license restrictions and requirements – especially if you’re running highly virtualized environments.
Get in touch today!
(Image from Torkild Retvedt)

MQ, The Digital Economy & You

Posted on | by Jeff Eveland

IBM MQ continues to evolve to meet the expanding needs of the digital economy. We encounter many organizations that have yet to take full advantage of the capabilities they’ve invested in their MQ backbone. Learn about the new MQ physical appliance, as well as the MQ virtual appliance (available exclusively from TxMQ). Other topics include secure transfer, cloud messaging and more.

How IBM MQ v8 Powers Secure Cloud Integration

Posted on | by admin

In this quickly growing digital economy, where we have an increasing demand on things like security, cloud and mobility, IBM MQ has been growing to meet those demands. To pick two of the three topics, MQ v8 can deliver secure cloud integration straight of the box.
It is important to know what type of cloud are we’re really talking about. Are you talking about moving all of your services into the cloud – even your virtual desktops? Or are you talking about a hybrid cloud where with a mix of cloud computing supplementing your own services? Or are you talking about a private cloud, where you’ll have segments of internal computing services totally isolated from general services. There are different considerations for each scenario.
Regardless of the type of cloud-computing services you’re using, you still need to integrate these services, and you really need to ensure that your integration has security, data integrity and the capability of sending messages once-only with assured delivery. Cloud can’t provide that. MQ can and does. And it does if out of the box with several recent enhancements to ensure secure integration.
With the digital economy, we’re all sharing all this data, including personal data, banking and health data. We need to keep this data secure when it’s being shared, and control who has access to it. Then of course there’s the large compliance piece we need to meet. How does MQ meet all these demands? The answer is authentication, and MQ’s solution is still the same as being asked for ID of proof at the post office when you go to pick up a package. MQ v8 has been enhanced to support full user authentication right out of the box. No more custom exits and plugins.
For distributed platforms, you have local OS authentication, or you can actually go to centralized data. For z/OS you’re still focused on local authentication.
And this next point is important: MQ for quite some time has supported certificate authentication of applications connected to MQ services. But this always meant that the public MQ key had to be shared with everyone. MQ now has been enhanced to support the use of multiple certificates for authentication, securing of connections and encryption using separate key pairs. MQ still support SSL and TLS, although there are strong recommendations for switching from SSL to TLS based on the POODLE vulnerability.
(Image from mrdorkesq)

MQTT – Valuable Then and Now

Posted on | by Jeff Eveland

In 1999, Dr. Andy Stanford-Clark of IBM and Arlen Nipper invented a simple messaging protocol designed for low bandwidth, high-latency networks. They named it MQ Telemetry Transport, better known as MQTT. This pub/sub, lightweight protocol adds a heightened security element to messaging via highly unreliable networks. It requires minimal network bandwidth and device resources, while ensuring MQ’s noted reliability and assured delivery standards.
What makes MQTT still valuable today? This protocol has dramatic implications and growing use cases for the Internet of Things (IOT), where the world of connected ‘things’ connects an endless variety of devices, sometimes with minimal power availability. In other words, as all these devices “talk” via the Internet, the MQTT  transport protocol ensures that these conversations stay secure and private. In order to understand how MQTT can improve your company’s ability to navigate the digital economy, you’ll need to get more acquainted with the nuts and bolts of the protocol.
Standards: While in the process of undergoing standardization at OASIS, the protocol specification is openly published and is royalty free.
SCADA and MQIsdp: SCADA Protocol and MQ Integrator SCADA Device Protocol are both old names for what is now known as MQTT.
Security: You can pass a user name and password with a MQTT packet in V3.1 of the protocol. Encryption is independent of MQTT and typically handled with SSL, though this does add network overhead. Keep in mind there are other options outside of the base protocol.
Implications and use cases: One of the oft-cited use cases, likely due to the underlying popularity of the application, is the Facebook messaging application.
When Facebook engineer Lucy Zhang was looking for a new messaging mechanism for their app, she knew she had to address bandwidth constraints, power consumption and platform variety (iOS, Android, Windows, etc). She turned to MQTT. While a truly innovative use for the protocol, this type of messaging isn’t the most typical use case.
M2M: MQTT’s true power lies in machine-to-machine communication. It’s specification to cover device communications enables any device to communicate information to any other system or device. Smart meters are an excellent example of an MQTT use case. This lightweight messaging protocol excels in communication among multiple sensors, often in remote locations, with limited power and inconsistent network connectivity.
In the case of smart grids, utilities companies can use MQTT to better predict where crews need to be in advance of weather events. In addition, transportation authorities can monitor road conditions to modify routes in advance of storms or accidents and detour commuters around construction sites.
Conclusions: MQTT has only recently come into its own as a mature, supported, reliable transport protocol to enable communication in a truly connected world – a world where meters can feed their data into analytics systems, combining with other data like weather information or social media trends, to perform predictive analytics.
TxMQ is working with a number of companies on next generation use cases for MQTT that better drives the digital economy, improves outcomes in healthcare, enhances lives and improves our planet for all of us.
Get in touch today for information on how we can partner with you on your digital evolution.
Chuck Fried is the President and CEO of TxMQ, a Premier IBM Business Partner supporting customers in the US and Canada since 1979.   chuck@txmq.com

MQTT Repositories Review – Mosquitto, MessageSight & More

Posted on | by admin

In my previous blog (Rigorous Enough! MQTT For The Internet Of Things Backbone), I presented the MQ Telemetry Transport (MQTT) protocol, which helps provide the required communication for smart devices. But without a broker repository or destination to support the protocol, MQTT can’t complete its mission.

In this article, I’ll first review one of the open-standard MQTT repositories called Mosquitto, and then cover IBM MessageSight. In future blogs I’ll present additional information on both the security component and additional broker functionality.

Mosquitto is an open-source (BSD-licensed) message broker that implements the MQTT protocol versions 3.1 and 3.1.1. It provides a lightweight server implementation of the MQTT and MQTT-SN protocols, written in C, so it can run on machines that can’t run a JVM.

Mosquitto regularly has an executable in the order of 120kB that consumes around 3MB RAM with 1,000 clients connected. There have been reports of successful tests with 100,000 connected clients at modest message rates.

In addition to accepting connections from MQTT clients, Mosquitto can bridge to other connected MQTT servers, including other Mosquitto instances. It’s thus possible to architect MQTT server networks, and pass MQTT messages from any network location to any other.

A second repository for MQTT is IBM MessageSight, which is built for high performance to offer persistent, transactional messaging. The hardware is 2U form factor. IBM MessageSight includes built-in security to enable integration with external Lightweight Directory Access Protocol (LDAP) security systems. MessageSight also offers Transport Layer Security (TLS), Secure Sockets Layer (SSL), FIPS 140-2, NSA Suite B ciphers and Level 1 secure Crypotgraphic Store securities.

Fine-grained messaging-authorization policies restrict access based on combinations of: user or group, client identifier, protocol, network interface, listening address and/or port, client IP address or range and destination topic and queue name.

The MessageSight repository supports connectivity to WebSphere Message Broker via JMS and/or MQTT nodes. It also integrates with Java environments and with rich HTML5-based web applications. Additionally, MessageSight allows development of interactive mobile-messaging applications with IBM Worklight Studio Developer, which delivers:

  • Friendly APIs and libraries
  • MQTT clients and libraries for a variety of platforms (C- and Java-based APIs)
  • Libraries for Google Android and Apple iOS
  • JMS client
  • JavaScript API for HTML5-based applications
  • PhoneGap MQTT plugins with JavaScript API for use with IBM Worklight
  • Apache Cordova
  • Adobe PhoneGap

MessageSight also offers simple and scalable management through policies. A single user ID is defined on the queue manager for IBM MessageSight, which enables a business to sense and respond to data coming from the edge of the enterprise. IBM MessageSight offers high availability with either an active or passive standby.

There are several public repositories that include Hive MQ, which provides a repository that anyone can engage with. In addition, there is cloudMQTT, which is a repository hosted in the cloud. There are other implementations of the broker space, namely gnatMQ, which is an implementation of MQTT but specifically for.Net, and ActiveMQ, which is a product of the Apache group.

Rigorous Enough! MQTT For The Internet Of Things Backbone

Posted on | by admin

The topic of mobile devices and mobile solutions is a hot one in the IT industry. I’ll devote a series of articles to exploring and explaining this very interesting topic. This first piece will focus on MQTT for the Internet of Things – a telemetry functionality originally provided through IBM.
MQTT provides communication in the Internet of Things – specifically, between the sensors and actuators. The reason MQTT is unique is, unlike several other communication standards, it’s rigorous enough to support low latency and poor connectivity and still provide a well-behaved message-delivery system.
Within the Internet of Things there’s a universe of devices that provide inter-communication. These devices and their communications are what enables “smart devices,” and these devices connect to other devices or networks via different wireless protocols that can operate to some extent both interactively and autonomously. It’s widely believed that these types of devices, in very short time, will outnumber any other forms of smart computing and communication, acting as useful enablers for the Internet of Things.
MQTT architecture is publish/subscribe and is designed to be open and easy to implement, with up to thousands of remote clients capable of being supported by a single server. From a networking standpoint, MQTT operates using TCP for its communication. TCP (unlike UDP) provides stability to message delivery because of its connection-oriented standard. Unlike the typical HTTP header, the MQTT header can be as little as 2 bytes, and that 2 bytes can store all of the information required to maintain a meaningful communication. The 2 bytes store the information in binary using 8 bits to a byte. It has the capability to add an optional header of any length. The 2 bytes of the standard header can carry such information as QOS, type of message, clean or not.
The quality-of-service parameters control the delivery of the message to the repository or server. The options are:

Quality-Of-Service Option Meaning
1 At most once
2 At least once
3 Exactly once

These quality-of-service options control the delivery to the destination. The first 4 bits of the byte control the type of message, which defines who’ll be interested in receipt of these messages. The type of message indicates the topic of the message, which will manage who receives the message. The last element will be the clean byte, which like the persistence in MQ will determine whether the message should be retained or not. The clean option goes a step further in that it will also tell the repository manager whether messages related to this topic should be retained.
In my next blog I’ll discuss the broker or repository for these messages. There are several repositories that can be used, including MessageSight and Mosquitto among others. The beauty of these repositories is their stability.
(Photo by University of Liverpool Faculty of Health & Life)