“Thou Shalt Not Run External Reports Against The Production Database” – ReadyERPSolutions.com, Chapter 3 Verse 1

For those of you running M3 on-premise with SQL Server, creating your own mirrored database for reporting purposes is easier than you might think (sorry M3 Cloud users, you’ll have to sit this one out since you don’t get SQL access).

The benefits of setting up a replicated database for reporting purposes are important and straightforward: basically, you avoid the risk of queries slowing down your production database server, which won’t just annoy your users, but could also impact the completion of transactions by the business engine.

I’ve had one client in particular where users gave up on doing anything useful before 10:00 a.m., because overnight reports were still typically running into the morning, slowing down M3 to the point where users couldn’t perform even basic functions.

But let’s get right to the action and show how to set this up…

Setting Up A Mirrored M3 Reporting Database

Note: There’s a document on Infor Xtreme titled “M3 Business Engine and Microsoft SQL Server – Best Practices” which I’d highly recommend reviewing for your combination of M3 and SQL Server version. We worked from the M3 BE 15.1.x/SQL Server 2014 document published in July 2016.

The basic idea here is that we will use SQL Server’s Replication functions to maintain the reporting database. It all starts with a snapshot which is brought over to your reporting server, and after the snapshot is taken transactions are logged and passed over in near real-time (in our experience, almost always less than 5 seconds).

The M3 Production database will be considered a Publisher in SQL Server, while your reporting database is termed a Subscriber. In this example I will create a replica from our TST database to a reporting database called M3TSTRPT.

Step 1: Create the reporting database

This part is real easy – just go into the SQL Server where you want reporting to be performed (NOT YOUR M3 PRODUCTION DATABASE SERVER), and right-click on Databases to create a New Database. Name it however you like (I’m going with M3TSTRPT here) and hit OK.

Next, it is important that you set the collation of the reporting database to the same as it is for M3, i.e. Latin1_General_BIN2 in our case. Immediately after creating your database, go into the Properties for it, and look for collation under the Options section. You want to get this set correctly before we start the replication process.

Step 2: Set up the Publisher

Now we need to designate our M3 database as a Publisher. Log into the SQL Server where your M3 database resides, and expand the Replication section. Right-click on Local Publications in order to create a New Publication. This will launch the New Publication Wizard, so let’s walk through those panels together.

1. First you are asked to designate a Distributor, which can be a server in the middle between the Publisher and Subscriber, which stores and transmits information. In our case, we allowed our M3 database to act as its own Distributor (the default option).
2. Do you want the SQL Server Agent service to start automatically? Yes, yes you do.
3. Next you specify the folder where the snapshot should be placed for the Subscriber to pick up. Since the Subscriber is on another server, this needs to be a network shared folder.
4. Now we select the database which will publish. Typically this will be M3FDBPRD, but since this example uses a TST environment, I am using M3FDBTST.
5. Publication Type: this is a biggie! Here we will select “Transactional Publication”, which operates as described earlier. A snapshot of the database is sent over to start with, and following that, incremental updates are passed along as they occur.
6. Articles: here you can select specific Tables and Views to bring over to your reporting database. I highly recommend you choose your tables based on reporting requirements, rather than just click the checkbox for everything, which could potentially cause performance issues. After all, you don’t need data from the various working tables in M3 such as FCR040. If you don’t get all the tables you need in this step, don’t worry! It is easy to add new ones later on (maybe I’ll cover that process in another article here). For this example I’m just going to bring over the user table CMNUSR.
7. There is a step where you can filter which rows are brought over in the replication process. If, for example, you had multiple companies in M3 and wanted to set up different reporting databases for each one, you could accomplish that by filtering the records you bring across by CONO. Typically, however, you’ll skip this step.
8. Next up, would you like to create a snapshot immediately, or schedule it for a later time? The main reason to schedule it later is if you plan to make changes to the snapshot before flipping the switch for replication (for example, you are waiting to get the final list of Articles you want to import in Step 6).
9. Agent Security: The SQL Server Agent will run all these processes, so you need to provide it with a user ID/password that has appropriate authority. In my example, I use a Windows account that has admin authority to the servers and folders in question, and for the “Connection to Publisher” I choose “Impersonate the process account.”
10. Next, you have a choice as to whether to create the Publication immediately (usually that’s a yes, unless you don’t have your final list of Articles yet), and whether or not to create a script that follows those same steps (in case you plan on doing this a lot, which I doubt).
11. Lastly, give this Publication a name! I’m calling this ReplDemo.

Assuming you told SQL Server to create the Publication immediately in Step 10, you’ll see a pop-up that shows the progress through the various steps of creating the publication and the initial snapshot.

Step 3: Subscribe

Now that we have a Publisher established, we need to tell our reporting database to become a Subscriber. Switch over your SQL Server connection to your reporting database, expand the Replication tab and use a right-click to create a New Subscription.

Once again, you’re led into a Wizard, so here we go:

1. Select the Publisher. From the drop-down list, select “Find a SQL Server Publisher”, which then prompts you to log into your M3 database server and browse for the Publication you just set up.
2. For the Distribution Agent Location, select the bottom option “Run each agent at its Subscriber”. This reduces the workload on your M3 database server, which is the whole point of this exercise, right?
3. Next, select which database will get updated via this Subscription… in this case that’s M3TSTRPT (the one we created in Step One).
4. Next, set the security access for the connections for both the Subscriber and Distributor processes. Much like the earlier bit, I use a Windows account with admin access, and for the Distributor connection I tell it to impersonate the process account.
5. You can schedule how often you want the Subscriber to run. Select “Run continuously” for those near real-time updates from M3.
6. Next, do you want to launch this immediately, or just create a script and run it later?
7. Also, do you want to actually create the Subscriber now, and/or create a script to use later?
8. Click Finish and off you go!

A pop-up will show the progress through the steps of creating the Subscription and starting the Snapshot Agent.

Once this is complete, we have one last change to make. On your reporting server, under Replication/Local Subscriptions, pull up the Properties for your new Subscription. Look for the Snapshot section, which probably reads “Default”. Use the drop-down to change that to Alternate Location, and specify the shared folder in the box immediately below (\\share\folder).

Step 4: Launch the Replication

At this point, if you open up your reporting database you shouldn’t see any File Tables in there yet. So let’s kick off the process!

Go over to your M3 database, browse under Replication/Local Publications, and use a right-click to Reinitialize. That should launch the creation of a new snapshot, and start the replication process running continuously.

Depending on how many tables you are bringing over, and their size, you should begin to see tables populate in your reporting database.

As a quick test I’ll pull up my own user ID record to see how it looks in the reporting database, and then go into M3 and make a change (such as inserting “TEST” at the beginning of my name). Within a few seconds of hitting Enter in M3, I refresh the query on my reporting database and sure enough, the update comes across!

Note: If you don’t see data showing up, check the Job Activity Monitor on both servers to see if the job ran into a problem. Most likely you have an issue with the security access, or perhaps the folder location where the snapshot is being dropped off and picked up.

Next Up: Monitoring Status, Adding Tables, & More

This should be enough to get you started with the process of creating a reporting database in SQL Server for M3. I’ll follow up with more on how we can monitor the status of the replication process, make changes as needed (typically adding more tables), and more.

I highly recommend you try this out first against your test environment to make sure you get all the steps right before implementing against PRD.

Happy Reporting!

Note: If you enjoyed this post and could use some help or assistance with Infor M3 (functional or technical), I am available for part-time after hours consulting. Please send a note through my Contact page or connect with me on LinkedIn for further details!

The post Building a Mirrored Database for M3 Reporting appeared first on Ready ERP Solutions.

Sample SQL from a report I once wrote to help with 1099 processing

One of the more amusing points that comes up when talking about the features of Infor M3 is the number of reports that come with the system and are available “out of the box”.

In my experience of many years and many M3 implementations I can say that very few customers use more than a handful of those “canned” reports, and without exception every M3 customer ends up building their own suite of reports, for both daily operational use and management oversight.

Those basic M3 reports typically go through StreamServe and allow users to either receive a PDF via email or a paper printout, both formats which aren’t easy to work with when analyzing large amounts of data. If you have hundreds or even thousands of records to work with, spreadsheets remain a vital tool in most environments.

Once the decision has been made to create a reporting suite against Infor M3, there are some important considerations to keep in mind, however! Be careful to follow these Commandments and save yourself a lot of heartache…

1. Thou Shalt Not Run External Reports Against The Production Database

This is a big big big big “NO”, and should mark the fundamental technical requirement of your external M3 reporting solution. The M3 Grid manages the interaction between the Business Engine and the database, and if you throw database queries from external report users into the mix, you run the risk of seriously reducing system performance for your users.

NOTE: This recommendation does not apply to reporting options within the M3 solution, such as a well-configured List Panel or the use of Ad Hoc Reports, which are managed by the M3 Grid and cooperate nicely with other processes.

I recall one client who regularly experienced slow system performance each morning, as various Crystal Reports ran to completion while users went for an extra cup of coffee or found other things to do rather than try to work within M3. It was one of those problems that started out as a minor annoyance but became worse over time.

The solution? We used BPW (Business Performance Warehouse) to define a datamart that copied selected tables over from M3 each night, and pointed the reports to run off the datamart instead. There are a number of other options you can use to create a parallel reporting database as well, particularly if you are running on SQL Server.

Not only did this allow users to be productive right from the start each morning in M3, but I imagine they saved a bit of coffee expense, too.

2. Honor Thy Users’ Requests For Spreadsheets

One of the big frustrations many M3 users have is the lack of an easy way to get reports in spreadsheet form from the system. Yes, you can dump the contents of a panel into Excel, and there is a cumbersome process that can turn StreamServe output into a spreadsheet, but these are far from ideal.

The reality is that for many situations, users need to be able to work with hundreds or even thousands of records at a time, and spreadsheets facilitate that kind of analysis far better than a PDF or other reporting format.

A tool like Crystal Reports or Microsoft’s SQL Server Reporting Services (SSRS) provide the flexibility for users to get their information in the format that fits them best, whether a PDF or spreadsheet.

3. Thou Shalt Write Proper SQL For Crystal Reports

One issue for you Crystal Reports users is to make sure that report writers understand the M3 database structure and write SQL statements to retrieve only the appropriate information for a given report, and no more!

If you use Crystal’s Select Wizard to pick which tables you need, and then apply filters to determine which records will show on a report, Crystal Reports will try to load those tables in their entirety and then filter down to show only the required lines. This is tremendously inefficient, both in terms of client performance and the amount of network traffic needed to run a particular report.

For example, if you have a daily Open Orders report for your purchasing department to monitor, using the basic Select Wizard in Crystal will cause entire purchase order tables (as well as any other tables you are linking to) to be brought over to the machine running the report, and then that machine will crank through the data and figure out which PO’s are open and which are not.

Instead, report writers should craft a proper SQL statement that retrieves only the data that is needed. This allows the database server to do the work that database servers are designed to do – grab the specific records and fields you need, and then only transmit that information across your network to the Crystal Reports application, which can then do the work of formatting things nicely and presenting the report.

4. Create SQL Views To Define Common Business Terms

Quite often you may need to develop a set of reports that offer different slices and dices over the same data, such as Open Order reports that may be broken out by product line, vendor, facility, etc. General Ledger data is another area where the same records and fields may need to be presented in a variety of ways.

In those cases, defining a View in your SQL reporting database can provide a shortcut for the development of most of those reports. That view can define standard record selection criteria and the fields needed across a family of reports, and be re-used in multiple queries. This reduces the time spent creating custom queries for each report, and avoids the risk of error when reinventing the wheel with every new report.

5. Custom M3 Reporting Is A Necessary Investment

An ERP system like Infor M3 doesn’t just serve as the mechanism that enforces your business rules, it also needs to grow and adapt as your business changes. Your reporting requirements will follow that change as well, so prepare yourself by making sure you have the necessary expertise to present whatever information your key business users need.

If you need assistance with reporting against Infor M3, head over to the Contact page and send a note! I can not only help with designing and implementing a custom reporting solution, but can coach up your technical staff so they can handle your reporting needs going forward.

The post The 5 Commandments of M3 Reporting appeared first on Ready ERP Solutions.