Monday, 30 July 2018

Oracle error -29548: ORA-29548 Solution (FNDCPASS Error)

Hey DBA-Mates,
Hope you all are doing Great in success life…
Friends, we faced this issue while changing the APPS password in R12.1.3 through FNDCPASS.
We are not sure, have you guys have faced this issue or not but hope this may useful here if anyone face this issue in future. As we have searched in Google or metalink and we hit many pages but didn’t worked for us. After that we decided to share with you all.
Please find the below details as per Real scenario:

$ . ./ora-data.blogspot.com.env

$ echo $TWO_TASK
ORA-DATA

1. Changing APPS password:

$ FNDCPASS apps/<prod_apps_password> 0 Y system/dev0dba SYSTEM APPLSYS <test_apps_password>

Log filename : L2230661.log
Report filename : O2230661.out

2. Changing SYSADMIN password:

$ FNDCPASS apps/<test_apps_pass>  0 Y system/dev0dba USER SYSADMIN <sysadmin_pass>
APP-FND-01564: ORACLE error 1017 in AFPCOA
Cause: AFPCOA failed due to ORA-01017: invalid username/password; logon denied
.
The SQL statement being executed at the time of the error was:  and was executed from the file .
$

Here, above error is not the main error because it clearly says invalid username and password. It means password was not changed.

So, we checked the APPS password change log file and we found the main error below error:

$ cat L2230661.log
+---------------------------------------------------------------------------+
Application Object Library: Version : 12.1

Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved.

FNDCPASS:
+---------------------------------------------------------------------------+

Current system time is 15-JUL-2018 08:42:51

+---------------------------------------------------------------------------+

Working...
Oracle error -29548: ORA-29548: Java system class reported: release of classes.bin in the database does not match that of the oracle executable - USER ( APPS ) has been detected in FND_WEB_SEC.GET_OP_VALUE.

+---------------------------------------------------------------------------+
Concurrent request completed
Current system time is 15-JUL-2018 08:42:51

+---------------------------------------------------------------------------+
$
Solution:
For this we have to run below these 2 scripts as SYSDBA:

SQL> @rmcorejvm.sql
SQL> @corejvminst.sql

$ cat rmcorejvm.sql
connect / as sysdba
set echo on
set serveroutput on
select * from v$instance;
select owner, status, count(*) from all_objects
where object_type like '%JAVA%' group by owner, status;
execute rmjvm.run(false);
shutdown immediate
set echo off
spool off
exit
$
$ cat corejvminst.sql
set serveroutput on
set echo on
startup mount
alter system set "_system_trig_enabled" = false scope=memory;
alter database open;
select owner, status, count(*) from all_objects
where object_type like '%JAVA%' group by owner, status;
create or replace java system
/
shutdown immediate
set echo off
spool off
exit

$
Hope this may useful and helpful. Please let us know for any suggestions @Contact US

Some more important links:

Regards,

Monday, 16 July 2018

FNDCPASS Command to Change APPS Password in R12.1.3

Hey DBA-Mates,Hope you all are doing great.
Recently, one of my junior team mates learned How to change APPS Password in R12.1.3. He is a junior and newly joined.

We asked him to change the APPS and SYSADMIN password after the clone.
But he was not done earlier and asked me some basics questions and its good. Even he google and checked but he was having some doubts.
He asked me some basics questions as:
1. Will we run FNDCPASS from SQL> prompt or $ prompt?
2. Apps Services should be UP or Down?

He knows the FNDCPASS uses to change the APPS user password but he have not done yet, so he got confused.
Then I thought it may be basics and easy lets share with you all guys.

Please find the below Steps:
1. FNDCPASS should be run from $ prompt.
2. Apps Services should be Down.
3. After changing Apps password through FNDCPASS, we should run autoconfig.
4. APPS and APPLSYS password will be same.

Steps to change the Apps password through FNDCPASS:
1. Run the env file.
$ . ./ora-data.blogspot.env

2. Cross check for correct server/host.
$ echo $TWO_TASK
ORA-DATA

3. Changing the APPS password through FNDCPASS:

$ FNDCPASS apps/<apps_oldpassword> 0 Y system/<system_pass> SYSTEM APPLSYS <apps_newpassword>
Log filename : L2331757.log


Report filename : O2331757.out
$

4. Always check the log because sometimes it will through error which you cannot know without checking log file.

$ cat L2331757.log
+---------------------------------------------------------------------------+
Application Object Library: Version : 12.1

Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved.

FNDCPASS:
+---------------------------------------------------------------------------+

Current system time is 15-JUL-2018 10:15:50

+---------------------------------------------------------------------------+

Working...

FNDCPASS completed successfully.

+---------------------------------------------------------------------------+
Concurrent request completed successfully
Current system time is 15-JUL-2018 10:15:58

+---------------------------------------------------------------------------+
$

5. Changing the SYSADMIN password :

$ FNDCPASS apps/<apps_newpassword>  0 Y system/dev0dba USER SYSADMIN <password for sysadmin>
Log filename : L2471757.log


Report filename : O2471757.out

6. Now check the logfile.

$ cat L2471757.log
+---------------------------------------------------------------------------+
Application Object Library: Version : 12.1

Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved.

FNDCPASS:
+---------------------------------------------------------------------------+

Current system time is 15-JUL-2018 10:16:42

+---------------------------------------------------------------------------+

Working...
Password is changed successfully for user SYSADMIN.
FNDCPASS completed successfully.

+---------------------------------------------------------------------------+
Concurrent request completed successfully
Current system time is 15-JUL-2018 10:16:46

+---------------------------------------------------------------------------+
$

Note: Once completed , please check and connect with APPS user with new password then start the services.

Regards,
ora-data Team

Wednesday, 11 July 2018

R12.2 Cloning Steps

Hey DBA-Mates,
Hope you all are rocking in APPS DBA because we DBAs always rock in case of work.. :D, just kidding.

Friends, when I searched in google almost people posted only for Apps tier cloning , what about DB tier which no one has mentioned. So, I thought let me cover that.

From long time, I was thinking for cloning as per real time scenario. But due to lack of time, I was not able to do so…

R12.2 CLONING STEPS:
Source: PROD
Target: TEST

1. Pre-requiste TASK AT PROD Instance PROD (Before Backup check space)
Please check the space for DB Backup, Conc Backup and App Backup as per your environment on server.

2. Preclone & Cold Backup on PROD Source                       
Prepare the source system database tier for cloning

$ cd $ORACLE_HOME/appsutil/scripts/PROD_hostname
$ perl adpreclone.pl dbTier

Prepare the Source System Application Tier for cloning
$ echo $FILE_EDITION  ==> Should return run

$ cd <INST_TOP>/admin/scripts
$ perl adpreclone.pl appsTier

Verification of adpreclone.pl done in all Tiers.
AT DBTier
Under cd $ORACLE_HOME/appsutil/clone below 2 files should be latest timestamp:
ls -lrtd jlib
ls -lrtd data
Same as above data => stage , below 2 files should be with current timestamp.
ls -lrt adcrdb.zip
ls -lrt addbhomsrc.xml

At AP(MT)
Here, we check the clone directory under $COMMON_TOP and for more confirmation we can check below two files:
cd $COMMON_TOP/clone
ls -lrtd jlib
ls -lrtd appl

As of Release 12.2, the adpreclone.pl process on the Application Tier creates a complete compressed archive of the Oracle Fusion Middleware such as:
Oracle WebLogic Server home,
Oracle Web Tier Utilities home,
Oracle Common Utilities home and
The Oracle E-Business Suite home.

Now, Shutdown PROD Completely [ because we are using cold backup]

cd $ADMIN_SCRIPTS_HOME
$./adstpall.sh

We need to take COLD Backup of PROD database and application RUN_BASE files system:

From DB Tier files need to be copied:

/u01/PROD/oradata/data
/u01/PROD/oradata/11.2.0 

From APPS Tier files need to be copied:

$RUN_BASE/EBSapps/appl
$RUN_BASE/EBSapps/comn
$RUN_BASE//EBSapps/10.1.2

Copy/Restore (unzip) to TEST by using cp command as per file structure and directory.
For Example:

$cp -R /u01/PROD/oradata/data /u01/TEST/oradata/data

Start the PROD Services because we don’t need.

3. Let’s start DB Clone:

Log on to the Target System as the ORACLE and Run the adcfgclone on the DBTier with dbTier option:

$ cd $ORACLE_HOME/appsutil/clone/bin
$ perl adcfgclone.pl dbTier

After adcfgclone.pl at DB-TIER please check whether DB services and DB listener is up and running. Then Proceed to next step.
Check DB Status and Create Temp Tablespace(if Not exists).

select count(*) from v$datafile; ==> should match as source count

Select distinct status from v$datafile;
Output as below shown:
SYSTEM
ONLINE

select file_name , tablespace_name , bytes/1024/1024 from dba_temp_files;
Example to add:
ALTER TABLESPACE TEMP ADD TEMPFILE '/u01/TEST/oradata/data/temp01.dbf' SIZE 1100M REUSE AUTOEXTEND OFF;

Clean FND_NODES table
Connect apps/<passwd>
EXEC FND_CONC_CLONE.SETUP_CLEAN;
commit;

Run Following command to close open mails:
sqlplus apps/appspwd

SQL>Update WF_NOTIFICATIONS set status = 'CLOSED', mail_status = 'SENT' where status = 'OPEN' and mail_status = 'MAIL';

Run autoconfig in DB tier and check whether the DB entry is updated in FND_NODES table.

$cd $ORACLE_HOME/appsutil/scripts/$CONTEXT_NAME
$adautocfg.sh

After Autoconfig runs successfull check following entry, it should not point to PROD :
sqlplus apps/appspwd
SQL> select node_name,support_db,last_update_date from fnd_nodes;

4. Application Tier Clone TEST

Run adcfgclone.pl on Apps Tier on "Run Edition File System".
Log on to the "Run Edition File System" in the Target System as the applmgr user:
$ cd <COMMON_TOP>/clone/bin
$ perl adcfgclone.pl appsTier

Start the application Tier using adstral.sh on RUN FILE System
cd $ADMIN_SCRIPTS_HOME
adstrtall.sh apps/appspwd

Run adpreclone on TARGET(TEST) RUN FILESYSTEM
We have to run adpreclone.pl on the "Run Edition File System" in the Target System. [New in R12.2]

Note: We are not Running Autoconfig because adcfgclone.pl will run internally autoconfig.

On TARGET (TEST) RUN FILE System
$ cd <INST_TOP>/admin/scripts
$ perl adpreclone.pl appsTier

SHUTDOWN THE APPLICATION which is running on RUN FILESYSTEM
cd $ADMIN_SCRIPTS_HOME
adstpal.sh apps/appspwd
Weblogic admin password:

COPY THE TARGET RUN FILESYSTEM to TARGET PATCH FILESYSTEM
Now, CONFIGURE TARGET PATCH FILESYSTEM

Run adcfgclone.pl on Apps Tier on "Patch Edition File System"
Log on to the "Patch Edition File System" in the Target System( as the APPLMGR user) and enter the following commands:

$ cd <COMMON_TOP>/clone/bin
$ perl adcfgclone.pl appsTier

Note-1: It will ask for Context file name, provide the fs1 (Run Fs) contetxt file name.
Note-2: Port will be different for Run and Patch FS.

Once completed: Startup the application tier services of  TEST

cd $ADMIN_SCRIPTS_HOME
adstrtal.sh apps/password
Enter WLSADMIN password:

Once above all done successfully, please perform Sanity Check.[Here, I can provide your some points or steps very soon]

5. Post Clone steps:

Update profile options and FND_CONCURRENT_REQUESTS:

sqlplus apps/passwd
UPDATE FND_CONCURRENT_REQUESTS
SQL> update fnd_concurrent_requests set status_code='C', phase_code='C'  where phase_code='P';

Change Site level Profile option.

Execute the below sql for running programs.
a) Set Terminating or Running to Completed/Terminated
SQL>UPDATE fnd_concurrent_requests
SET phase_code = 'C', status_code = 'X'
WHERE status_code ='T'
OR phase_code = 'R'
/

b) Place Pending/(Normal/Standby) to On Hold
SQL>UPDATE fnd_concurrent_requests
SET hold_flag = 'Y'
WHERE phase_code = 'P'
AND status_code in ('Q','I')
/

Then check for Health check and sanity check. Login to front end and submit the request.

Some more useful links:
Please let us know for any suggestions either in comment box or contact us @https://ora-data.blogspot.com
Regards,

Sunday, 17 June 2018

How to Duplicate a database using RMAN backup

Dear DBA-Mates,
Hope you all doing Good!!!
We are back with RMAN database backup and RMAN duplicate database command which we were promised in our last post OracleRMAN Interview Questions and Answers.
Here, simply we are proving the commands for How to take RMAN back and then how to restore as RMAN Duplicate database.
Many of us think how to take rman backup and rman duplicate the database in real scenario because still there are some organizations where RMAN is not
using. But now a day RMAN is very necessary.

RMAN Important Terms:

DEVICE TYPE --> disk or tape or SBT
IMAGE COPIES -- simple copy of databse as cp command in linux
BACKUPSET --> backupsets is a logical entity to backup pieces as a tablespace to data files. Backup pieces are in an RMAN specific binary format.

Note: You cannot perform incremental backups with image copies but you can do so with backup sets.

FORMAT --> to store backup at specific location/speciafic format (date/time).

TAG --> to identify/alias name for backup.

AS COMPRESSED BACKUPSET --> binary( Oracle database files) compression of backupsets, performance overhead during backups and restores.

Steps to take RMAN Backup from database either PROD or STANDBY:
We should use STANDBY because performance will not effect on PROD.

RMAN BACKUP STEPS:
sql "alter session set nls_date_format=''dd.mm.yyyy hh24:mi:ss''";
RUN
{
configure controlfile autobackup on;
set command id to 'DBOnlineBackupFull';
ALLOCATE CHANNEL c1 DEVICE TYPE disk;
ALLOCATE CHANNEL c2 DEVICE TYPE disk;
ALLOCATE CHANNEL c3 DEVICE TYPE disk;
ALLOCATE CHANNEL c4 DEVICE TYPE disk;
ALLOCATE CHANNEL c5 DEVICE TYPE disk;

backup AS COMPRESSED BACKUPSET full database 
tag <backup_name_full
format '< backup location with format>/%d_%T_%s_%p_FULL' ;

backup archivelog all
tag <DB_ARCHIVE>  
format '<location with format>/%d_%T_%s_%p_ARCHIVE' ;

backup current controlfile
tag <DB_CONTROL 
format '<location with format>/%d_%T_%s_%p_CONTROL';

release channel c1;
release channel c2;
release channel c3;
release channel c4;
release channel c5;

}

After Backup completed, scp to the Target (Test) server and follow the below steps to DUPLICATE the database.

Now perform the RMAN Duplication as below.
Step 1) Login to Target (Test) database tier and source the Environment
Step 2) connect as sysdba , i.e. sqlplus ‘/as sysdba’
SQL> startup nomount;  ## it should start from spfile.
SQL>exit
$rman log=RMAN_dup_SID_timestamp.log
RMAN> connect auxiliary /
RMAN> run
 {
ALLOCATE AUXILIARY CHANNEL aux1 DEVICE TYPE DISK;
ALLOCATE AUXILIARY CHANNEL aux2 DEVICE TYPE DISK;
ALLOCATE AUXILIARY CHANNEL aux3 DEVICE TYPE DISK;
ALLOCATE AUXILIARY CHANNEL aux4 DEVICE TYPE DISK;
ALLOCATE AUXILIARY CHANNEL aux5 DEVICE TYPE DISK;

DUPLICATE DATABASE TO <DATABASE_NAME> BACKUP LOCATION '/<backup location>/' NOFILENAMECHECK;
}

Whereas:
NOFILENAMECHECK:
Suppose, If we want the duplicate filenames to be the same as the target filenames, and also if our databases are in different hosts, then we should specify the option as NOFILENAMECHECK.
<backup_name_full>
example: proddb_bkp_full
<backup location with formate>/%d_%T_%s_%p_FULL 
example: /rmanbkp/PROD/%d_%T_%s_%p_FULL' ;

You may like some more below:
Regards,
ora-data Team

Monday, 28 May 2018

How to apply CPU OCT 2015 patch on 10.1.3 in r12.1.3

Hey DBA-Mates…
Hope you all doing well. CPU patches is little bit different than other patches but as I always says we/you can do it just need time and patient. 
Recently one of my colleague applied CPU Oct2015 apps patches 10.1.3.5 in R12.1.3
This below steps which guided to him and patches applied successfully. 
So, I thought to share with you all. Hope this may useful and helpful in your career. 
It is almost from basic and each point is provided.
I would like to request one more thing, please always read and follow the patches either readme.txt or readme.html which is very important. So, let’s see the below steps:
Check for OS details for downloading correct patch:
OS version details:
$ uname -a
SunOS hostname 5.xx $$.$$ sun4v sparc sun4v

$ cat /etc/release
                            Oracle Solaris $$.$ SPARC
  Copyright (c) 1983, 2017, Oracle and/or its affiliates.  All rights reserved.
                            Assembled 05 October 2017
$ isainfo -b
64 (bit)

$ opatch version
Oracle Interim Patch Installer version 1.0.0.0.64
Copyright (c) 2011 Oracle Corporation. All Rights Reserved..
Oracle recommends you to use the latest OPatch version
and read the OPatch documentation available in the OPatch/docs
directory for usage. For information about the latest OPatch and
other support-related issues, refer to document ID 293369.1
available on My Oracle Support (https://myoraclesupport.oracle.com)

OPatch Version: 1.0.0.0.64

Note: Always read patch either readme.txt or readme.html and follow that.
Note: As per readme.html Opatch version should be min 1.0.0.0.63, here in our case it is 1.0.0.0.64

$ cd $INST_TOP/ora/10.1.3
$ pwd
/apps/ebs/inst/apps/TEST_hostname/ora/10.1.3

$ echo $ORACLE_HOME
/apps/ebs/apps/tech_st/10.1.2   ## it should be point to 10.1.3
$ ls -ltr
total 15
drwxr-xr-x   3 applmgr  oinstall    network
drwxr-xr-x   3 applmgr  oinstall    Apache
drwxr-xr-x   3 applmgr  oinstall    javacache
drwxr-xr-x   6 applmgr  oinstall    j2ee
drwxr-xr-x   2 applmgr  oinstall    deconfig
drwxr-xr-x   2 applmgr  oinstall    config
drwxr-xr-x   5 applmgr  oinstall    opmn
drwxr-xr-x   2 applmgr  oinstall    cfgtoollogs
drwx------   3 applmgr  oinstall    ccr
-rw-r--r--   1 applmgr  oinstall    TEST_hostname.env

$ . ./TEST_hostname.env

$ echo $ORACLE_HOME
/apps/ebs/apps/tech_st/10.1.3
$

Pre check as it is applied or not:
$ opatch lsinventory | grep 21845960  [Main Patch number which no need output]

$ opatch lsinventory | grep 21845942  [Sub-patches if applied should show in output, which we will check after applying the patches]

$ pwd
/apps/ebs/apps/tech_st/10.1.3/opatches/21845960

$ echo $ORACLE_HOME
/apps/ebs/apps/tech_st/10.1.3

$ echo $IAS_ORACLE_HOME
/apps/ebs/apps/tech_st/10.1.3
$ echo $TWO_TASK
TEST
$
As per readme.html these below path should be mentioned in $PATH
$ ls -ltr /usr/bin/ld
-r-xr-xr-x   1 root     bin        24688 Mar 24 10:29 /usr/bin/ld
$ ls -ld /usr/bin/ar
-r-xr-xr-x   1 root     bin        43904 Mar 24 10:29 /usr/bin/ar
$ ls -ld /usr/bin/nm
-r-xr-xr-x   1 root     bin        48128 Mar 24 10:30 /usr/bin/nm
$ ls -ld /usr/bin/make
-rwxr-xr-x   1 root     bin       212988 May 25  2017 /usr/bin/make
$
If it now included in PATH then need to export as below shown:

$ export PATH=$PATH:/usr/bin/ld:/usr/bin/ar:/usr/bin/nm:/usr/bin/make

$ echo $PATH
$ pwd
/apps/ebs/apps/tech_st/10.1.3/opatches/21845960
$ which opatch
/apps/ebs/apps/tech_st/10.1.3/OPatch/opatch
$ echo $ORACLE_HOME
/apps/ebs/apps/tech_st/10.1.3
$ echo $TWO_TASK
TEST
$ pwd
/apps/ebs/apps/tech_st/10.1.3/opatches/21845960
$ ls -ltr
total 70
drwxr-xr-x   4 applmgr  oinstall       5 Oct 13  2015 21845942
-rwxr-xr-x   1 applmgr  oinstall    2258 Oct 13  2015 remove_demo.sh
-rw-r--r--   1 applmgr  oinstall      67 Oct 13  2015 README.txt
drwxr-xr-x   5 applmgr  oinstall       7 Oct 13  2015 21815758
drwxr-xr-x   4 applmgr  oinstall       5 Oct 13  2015 9273888
drwxr-xr-x   5 applmgr  oinstall      10 Oct 13  2015 10011970
drwxr-xr-x   5 applmgr  oinstall       6 Oct 13  2015 10036362
drwxr-xr-x   5 applmgr  oinstall       6 Oct 13  2015 14123312
-rw-rw-r--   1 applmgr  oinstall   57215 Oct 19  2015 README.html
$
Check for Invalid Objects:

SQL> select object_name,object_type,owner from dba_objects where status='INVALID'
$ pwd
/apps/ebs/apps/tech_st/10.1.3/opatches/21845960
$opatch napply

Note: Here napply we are using for multiple patches to ally as shown above under the cpu patches there were many patches.

After that it will ask many times as Y/N, read once I always provided Y as shown below like:
Please respond Y|N > Y

Once OPatch succeeded Then check for applied or not as below command:
$opatch lsinventory | grep 21845942
  2) Patch  21845942 applied on Sun May 27 02:17:58 GMT-05:00 2018
      [ Bug fixes: 17232193 7215134 12434146 21845942 21843498 9173042 7457583 19647609 15930322 17232255 17988318 6406333 6855621 18272601 19952531 18855617 7576325 6768420 16264435 14003474 20034700 13564298 16021033 19288606 17232750 17250040 17643943 9481810 17664485 17631115 16802872 6768427 16193586 20900385 6032064 8806540 7632951 16021138 13564285 17664561 12434225 18746609 14251913 16802900 18272665 14110089 14010533 19947918 17329650 19319912 12959528 8800528 19320420  ]

$ opatch lsinventory | grep 21815758
  3) Patch  21815758 applied on Sun May 27 02:10:56 GMT-05:00 2018

$  opatch lsinventory | grep 9273888
  1) Patch  9273888 applied on Sun May 27 02:19:55 GMT-05:00 2018

      [ Bug fixes: 9273888  ]

$ opatch lsinventory | grep 10011970
  5) Patch  10011970 applied on Sun May 27 01:50:07 GMT-05:00 2018
      [ Bug fixes: 6912255 6790178 7428561 6433471 5397517 3645912 5724681 4473073 6682888 7157442 8416899 6080374 6471931 6912781 7552946 5453754 6018059 6150541 9336949 5999450 5740055 7021360 7335185 6713795 5477073 6134487 7196121 7430171 6080307 5259451 6655345 10011970 7146872 5697416 6024000 6153975 7353810 5932346 7519011 7195030 6844221 5979883 5486768 7127677 7229577 5985742 6852049 4635520 6917549 5871130 7004606 6237650 5573438 6681624 6772953 6816068 7574599 5763122 6526074 5902630 7113141  ]

$ opatch lsinventory | grep 10036362  [ There is no problem if not showing, during applying it will show as Opatch success just check alert log.]

$ opatch lsinventory | grep 14123312
  4) Patch  14123312 applied on Sun May 27 02:06:07 GMT-05:00 2018
      [ Bug fixes: 14123312  ]
$
Or

Note: After applying the above patch, please check the readme.html where is mentioned the Post steps.

Hope this may useful and helpful. Please let us know for any suggestions or correction in our either comment box or contact us @ora-data.blogspot.com

Some more useful links:





Regards,