Improve Your Data Quality, A starting point for your Data Management strategy – A Business Focused Approach
May 18, 2012 Leave a comment
Click the link/title to see the video.
Starting a data governance program…
August 5, 2011 Leave a comment
Enterprise Data management(EDM) refers to the ability of an organization to precisely define, easily integrate and effectively retrieve data for both internal applications and external communication. EDM emphasizes data correction, consistency, precision, granularity and meaning and is concerned with how the content is integrated into business applications as well as how it is passed along from one business process to another.
Data governance is the crux of any enterprise data management startegy. Data Governance refers to the overall management of the availability, usability, integrity, and security of the data employed in an enterprise. A sound data governance program includes a governing body or council, a defined set of procedures, and a plan to execute those procedures. In practical terms, that means putting personnel, policies, procedures, and organizational structures in place to make data accurate, consistent, secure, and available to accomplish business goals. Effective data governance makes meaningful and correct data available to business
and hence it makes business process more efficient by saving money, allowing re-use of data, and supporting enterprise analytics. However, data governance requires more than just a few members of the IT staff with a project plan. It requires participation and commitment of both IT and business management, as well as senior-level executive sponsorship and active consultation with various business communities of interest.
In my last company, data governance was planned, managed, and implemented through a three level structure:
- The Executive Data Governance Council provides strategic direction, ensuring that data governance efforts
address all relevant and mission-critical needs of the enterprise. It manages data governance as an integrated program rather than as a set of unconnected projects. - The Strategic Data Governance Steering Committee carries out plans and policies to implement guidance from the Executive Data Governance Council. It prioritizes data governance efforts and communicates with stakeholders, users, and other communities of interest.
- The Tactical Data Governance working group implements plans and policies developed by the EDM Governance team, and analyzes and resolves any tactical problems that arise.
Communication is very important for successful data governance. To succeed in a data governance program, management bodies and implementation team(s) must tell stakeholders what steps are being taken and why, must inform all relevant communities of interest about how data governance will benefit them, and must listen to stakeholders and communities of interest to incorporate their ideas and feedback into the data governance program. Input and feedback makes governance efforts more effective in achieving mission-critical goals and is vital for successful data governance.
Data Governance program needs continued interest and participation from business. Data owners should be from business side not from IT and they should be able to demonstrate the need, business value of data & ROI achieved by a data governance program.
A glance at SAP data migration methods….
July 4, 2011 3 Comments
What are the various methods available for SAP Data Migration? I studied few ongoing prominent SAP Data Migration projects and had a discussion with our Data Migration team. As per my understanding, there are three popular methods for SAP data migration from legacy systems and/or old SAP R/3 to new SAP ECC system.
- SAP Best Practices – Pre built contents based on SAP Data Services (ETL) that utilizes primarily IDOCs to load data into SAP.
- LSMW – A utility by SAP that utilizes flat files to load data into SAP
- Custom Developed Programs – Uses SAP BDC programs and flat files.
Each method has its advantages and disadvantages. I will discuss what I know about these methods, advantages and disadvantages of one method vs. another, challenges faced by clients by using any of these methods etc. In this blog, I will talk about SAP Best Practices. In subsequent posts, I will discuss LSMW, Custom Developed Programs, Advantages, Disadvantages, Challenges etc.
SAP Best Practices Method
Let’s talk about data migration from legacy(non-SAP) systems to SAP system. This includes new SAP customers as well as current customers who are bringing in new plants, new business units, etc., and need to convert data to a SAP ECC system. SAP Information Lifecycle Management (ILM) is used for system decommissioning or data retention and archival. It is beyond the scope of this discussion at this time.
This method utilizes loading of data into SAP primarily by IDOCs. SAP acquired Business Objects tools such as Business Objects Data Integrator ETL, Data Quality (First Logic) and bundled it together with a new avatar “SAP Data Services”. The core strength of Business Objects Data Services, earlier known as Business Objects Data Integrator ETL or Acta ETL has been tight integration with SAP. This ETL tool was primarily used for SAP data extraction since its inception in 1998 or so. I have seen the evolution of tool from Acta 1.1 to SAP Data Services XI 4.x. There are some other Business Objects software too used in migration such as Data Insight (Data Profiling tool), Metadata Manager (these two tools now known as Information Steward) and some reports, but SAP Data Services is where the bulk of the work takes place. For those who don’t know – Business Objects America acquired a company Acta Technology in 2002 or so and SAP acquired Business Objects Americas in 2007. Business Objects renamed the Acta ETL as Business Objects Data Integrator after Acta acquision and later SAP renamed it as SAP Data Services.
Acta also offered SAP Rapid Marts. Rapid Marts are out of box pre-packaged Acta ETL code and target database schema based on Oracle or SQL Server databases for extraction of data from various SAP modules such as SD, IM, FI, CO, GL, HR and so on. The value proposition of Rapid Marts has been that it gives a jump start to SAP customers in terms of getting data out of SAP quickly. Customers are generally able to leverage 65-70% of out of box Rapid Mart contents in its AS IS mode. Remaining contents can be easily customized based on customer’s SAP configuration etc. and generally entails addition of deletion of fields in tables in Rapid Marts, extraction of SAP custom table(s) if any etc. These Rapid Marts are standard SAP Data Mart offerings from SAP based on SAP Data Services now.
SAP has developed similar out of box SAP Data Services ETL codes for data migration to SAP based on standard SAP ECC Master data structures. These are called Best Practice(BP) Content for Data Migration. It is also known as SAP AIO BP, which is nothing but “SAP Business All-in-One” Best Practices. It is confusing to see so many new SAP terms but don’t let it scare you. SAP is pioneer in coming up with new buzzwords however core contents remain more or less the same behind the scenes.
The BP content for Data Migration can be found under the Data Migration, Cross-Industry Packages in Best Practices section in HELP portal. This content has everything you need to get started on migrating non-SAP data to an SAP system. The content includes the following: guides to install SAP data services and other components required for the migration, actual content to load that includes jobs to load data into SAP via IDOCs, mapping tools to help you map the non-SAP data to the IDOC structure, and some reports. It includes IDOC mapping and structures for objects like Material Master, Vendor Mater and Customer Master, Pricing, BOM, Cost element, Payables and Receivables contents. There are detailed word documents on each piece of content, for example a document on Material that is a 39 page word document, covering the IDOC structures, what you need to know, and how to map data to the structure.
SAP also provides standard data migration methodology, framework, templates, based on SAP Best practices and SAP Data Services. Methodology has components – Analyze, Extract, Cleanse, Validate, Upload and Reconcile legacy data into a SAP ERP environment.
This method of data migration using SAP Best Practices and IDOCs work very well in case no customization is required for data migration. What it means is that if a customer has standard SAP ECC vanilla implementaion, this method works just GREAT. For example, a SAP Best Practices per built job for material master loads the data as per standard ECC Material Master IDOC structure. In case customer needs more fields, or a custom table is to be loaded in Material Master, it is easy to modify or add to SAP Best Practices ETL code however along modify BP code will not suffice. Corresponding SAP IDOCs need to be modified or extended as well which may or may not be allowed by customer’s SAP Basis team. Customer will also need SAP ABAP/IDOC expertise on the project to modify IDOC structure. Many customers don’t prefer to modify standard IDOCs.
Another scenario where SAP Best Practice will not work is if there is no one to one mapping between the input and output data. In other words if master data element to be convereted into SAP ECC is dependent on more than one dimension of input data, SAP Best Practices will not work. Let’s take an example, if sales org A in legacy system is to be converted into sales org B in SAP ECC, SAP Best Practice will work great. However if there are three sales orgs A, B, C in legacy systems and there is needed only one sales org D in SAP ECC with value dependent on three dimensions such as Sales org, Plant, Country code in source data in legacy systems, SAP Best Practice can’t handle this conversion scenario at least as of today. In this case, a good amount of customization needs to be done in SAP Best Practices code, tables, scripts etc which may not be worth the efforts and may impact the integrity of SAP Best Practices contents dependent on the modified content/code.
A similar approach is taken for data migration from one or many SAP systems, Legacy System to SAP ECC system. In this option, maybe you have multiple SAP systems on different releases, so one on 4.6c, 4.7 and you want consolidate to a single ECC 6.0 system. You can use SAP Data Services to extract data from old SAP system, non-SAP system and use same methodology, framework and SAP Best Practices to load data into SAP ECC similar to what we discussed above.
To be contd…..
