高危风险，银河麒麟系统安装Oracle数据库遇到的严重问题！
-
前言：
由于不同产品国产化替代进度不同，导致适配上经常出问题，最明显的就是国产化操作系统替代的速度通常快于国产化数据库的替代，这就不得不面临一个问题，在先替换国产操作系统后，如何平滑的临时将国外商业数据库迁移到国产操作系统上？
以本文为例，对国产麒麟系统来讲，又分为中标麒麟和银河麒麟等，其中oracle又只兼容中标麒麟，如果想在银河麒麟系统安装oracle，通常会遇到一些问题。
关于Oracle数据库适配哪些国产操作系统，请看我的另一篇文章：
Oracle数据库适配哪些国产操作系统？
chenjuchao，公众号：IT小ChenOracle数据库适配哪些国产操作系统？
-
问题描述：
在银河麒麟Kylin Linux V10 (SP1)上静默安装Oracle 11.2.0.4.0软件后，发现 $ORACLE_HOME/bin 目录下 lsnrctl,tnslsnr,rman,sqlldr,expdp ...等很多文件大小为0，导致了很多命令无法正常执行。
除此以外，还有其他几个问题，请看我的另一篇文章：
银河麒麟系统安装ORACLE数据库问题处理
chenjuchao，公众号：IT小Chen银河麒麟系统安装ORACLE数据库问题处理
-
不靠谱的方案：
手动拷贝另一套没问题的数据库的相应文件，并授予相应的权限。
scp -P 22 sqlldr rman oracle lsnrctl expdp exp adrci tkprof tnsping tnslsnr ... oracle@CJC-DB-01:/home/oracle/tmp
复制到 $ORACLE_HOME/bin 目录下。
拷贝后，可以正常执行数据库命令，也可以正常继续后面的dbca操作。
但是这种方案有很大的安全隐患，lsnrctl等文件大小为0的原因没有找到，很可能还有其他安全问题，所以生产环境不建议这么操作。
-
问题原因：
具体还是要分析lsnrctl等文件大小为0的原因。
先检查下版本信息：
操作系统版本：
root@cjc-db-01:/tmp/oraclesoft_20211216#nkvers ############## Kylin Linux Version #################Release:Kylin Linux Advanced Server release V10 (Tercel)Kernel:4.19.90-23.43.v2101.ky10.x86_64Build:Kylin Linux Advanced Serverrelease V10 (SP1) (Tercel)-x86_64-Build20/20210518
ldd版本:
root@cjc-db-01:/root#ldd --versionldd (GNU libc) 2.28Copyright (C) 2018 Free Software Foundation, Inc.This is free software; see the source for copying conditions. There is NOwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.Written by Roland McGrath and Ulrich Drepper.
gcc版本:
root@cjc-db-01:/root#gcc --versiongcc (GCC) 7.3.0Copyright (C) 2017 Free Software Foundation, Inc.This is free software; see the source for copying conditions. There is NOwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
其中 ldd，gcc版本比redhat7.9上的ldd,gcc版本高。
检查静默安装日志:
执行安装，速度很快，不到2分钟安装完了，自动忽略了错误，之前以为是静默安装添加了-ignoreSysPrereqs,-ignorePrereq参数导致的，但是去掉这两个参数还是会自动忽略，反正你也别管安装的有没有问题，你就说快不快吧！
过程如下：
root@cjc-db-01:/tmp/oraclesoft_20211216#su - oracle -c "LANG=C; tmp/oraclesoft_20211216/database/runInstaller -silent -noconfig -showprogress -responseFile tmp/oraclesoft_20211216/install_database_rsp_file.rsp &" | tee tmp/oraclesoft_20211216/install_database.logStarting Oracle Universal Installer...Checking Temp space: must be greater than 120 MB. Actual 4638 MB PassedChecking swap space: must be greater than 150 MB. Actual 1461 MB PassedPreparing to launch Oracle Universal Installer from tmp/OraInstall2024-04-03_08-52-26AM. Please wait ...[WARNING] [INS-32055] The Central Inventory is located in the Oracle base. CAUSE: The Central Inventory is located in the Oracle base. ACTION: Oracle recommends placing this Central Inventory in a location outside the Oracle base directory.[WARNING] [INS-13014] Target environment do not meet some optional requirements. CAUSE: Some of the optional prerequisites are not met. See logs for details. tmp/OraInstall2024-04-03_08-52-26AM/installActions2024-04-03_08-52-26AM.log ACTION: Identify the list of failed prerequisite checks from the log: tmp/OraInstall2024-04-03_08-52-26AM/installActions2024-04-03_08-52-26AM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.You can find the log of this install session at: db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_08-52-26AM.logPrepare in progress................................................... 9% Done.Prepare successful.Copy files in progress................................................... 14% Done................................................... 20% Done................................................... 26% Done................................................... 31% Done................................................... 36% Done................................................... 41% Done................................................... 47% Done................................................... 52% Done................................................... 57% Done................................................... 63% Done................................................... 68% Done................................................... 73% Done................................................... 78% Done................................................... 83% Done...............................Copy files successful.Link binaries in progress...........Link binaries successful.Setup files in progress................................................... 88% Done................................................... 94% Done.Setup files successful.The installation of Oracle Database 11g was successful.Please check '/db/oracle/app/oracle/oraInventory/logs/silentInstall2024-04-03_08-52-26AM.log' for more details.Execute Root Scripts in progress.As a root user, execute the following script(s): 1. db/oracle/app/oracle/oraInventory/orainstRoot.sh 2. db/oracle/app/oracle/product/11.2/db/root.sh.................................................. 100% Done.Execute Root Scripts successful.Successfully Setup Software.
检查安装日志：
vi db/oracle/app/oracle/oraInventory/logs/silentInstall2024-04-03_08-52-26AM.log'......INFO: INFO: *********************************************INFO: INFO: Package: compat-libstdc++-33-3.2.3 (x86_64): This is a prerequisite condition to test whether the package "compat-libstdc++-33-3.2.3 (x86_64)" is available on the system.INFO: INFO: Severity:IGNORABLEINFO: INFO: OverallStatus:VERIFICATION_FAILEDINFO: INFO: -----------------------------------------------INFO: INFO: Verification Result for Node:cjc-db-01INFO: INFO: Expected Value:compat-libstdc++-33(x86_64)-3.2.3INFO: INFO: Actual Value:missingINFO: INFO: Error Message:PRVF-7532 : Package "compat-libstdc++-33(x86_64)" is missing on node "cjc-db-01"INFO: INFO: Cause: A required package is either not installed or, if the package is a kernel module, is not loaded on the specified node.INFO: INFO: Action: Ensure that the required package is installed and available.INFO: INFO: -----------------------------------------------INFO: INFO: *********************************************......
缺失如下安装包，银河麒麟官网没有找到这些包，尝试将redhat 7上的这些包考过了强制安装，依然没有解决问题。
compat-libstdc++-33(x86_64)-3.2.3elfutils-libelf(x86_64)-0.97elfutils-libelf-devel-0.97glibc-headers-2.3.4pdksh-5.2.14
继续查看安装日志：
silentInstall2024-04-03_11-11-38AM.logError in invoking target 'client_sharedlib' of makefile '/db/oracle/app/oracle/product/11.2/db/rdbms/lib/ins_rdbms.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'links proc gen_pcscfg' of makefile '/db/oracle/app/oracle/product/11.2/db/precomp/lib/ins_precomp.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'idg4odbc' of makefile '/db/oracle/app/oracle/product/11.2/db/rdbms/lib/ins_rdbms.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'itnsping' of makefile '/db/oracle/app/oracle/product/11.2/db/network/lib/ins_net_client.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'install' of makefile '/db/oracle/app/oracle/product/11.2/db/ctx/lib/ins_ctx.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'install' of makefile '/db/oracle/app/oracle/product/11.2/db/plsql/lib/ins_plsql.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'agent nmhs' of makefile '/db/oracle/app/oracle/product/11.2/db/sysman/lib/ins_emagent.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'all_no_orcl' of makefile '/db/oracle/app/oracle/product/11.2/db/rdbms/lib/ins_rdbms.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'install' of makefile '/db/oracle/app/oracle/product/11.2/db/network/lib/ins_net_client.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'install' of makefile '/db/oracle/app/oracle/product/11.2/db/sqlplus/lib/ins_sqlplus.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'clientonlyinstall' of makefile '/db/oracle/app/oracle/product/11.2/db/ldap/lib/ins_ldap.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'ioklist' of makefile '/db/oracle/app/oracle/product/11.2/db/network/lib/ins_nau.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'install' of makefile '/db/oracle/app/oracle/product/11.2/db/srvm/lib/ins_srvm.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'utilities' of makefile '/db/oracle/app/oracle/product/11.2/db/rdbms/lib/ins_rdbms.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'iokdstry iokinit' of makefile '/db/oracle/app/oracle/product/11.2/db/network/lib/ins_nau.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'install' of makefile '/db/oracle/app/oracle/product/11.2/db/network/lib/ins_net_server.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'itrcroute' of makefile '/db/oracle/app/oracle/product/11.2/db/network/lib/ins_net_client.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.Error in invoking target 'irman ioracle' of makefile '/db/oracle/app/oracle/product/11.2/db/rdbms/lib/ins_rdbms.mk'. See '/db/oracle/app/oracle/oraInventory/logs/installActions2024-04-03_11-11-38AM.log' for details.File not found: %fileName%The installation of Oracle Database 11g was successful.
有问题的文件：
/db/oracle/app/oracle/product/11.2/db/rdbms/lib/ins_rdbms.mk/db/oracle/app/oracle/product/11.2/db/precomp/lib/ins_precomp.mk/db/oracle/app/oracle/product/11.2/db/rdbms/lib/ins_rdbms.mk/db/oracle/app/oracle/product/11.2/db/network/lib/ins_net_client.mk/db/oracle/app/oracle/product/11.2/db/ctx/lib/ins_ctx.mk/db/oracle/app/oracle/product/11.2/db/plsql/lib/ins_plsql.mk/db/oracle/app/oracle/product/11.2/db/sysman/lib/ins_emagent.mk/db/oracle/app/oracle/product/11.2/db/rdbms/lib/ins_rdbms.mk/db/oracle/app/oracle/product/11.2/db/network/lib/ins_net_client.mk/db/oracle/app/oracle/product/11.2/db/sqlplus/lib/ins_sqlplus.mk/db/oracle/app/oracle/product/11.2/db/ldap/lib/ins_ldap.mk/db/oracle/app/oracle/product/11.2/db/network/lib/ins_nau.mk/db/oracle/app/oracle/product/11.2/db/network/lib/ins_net_server.mk/db/oracle/app/oracle/product/11.2/db/network/lib/ins_net_client.mk/db/oracle/app/oracle/product/11.2/db/rdbms/lib/ins_rdbms.mk
-
最终解决方案：
用上面有问题的文件为关键字，在MOS上搜索，最终通过ins_precomp.mk关键字，定位到了一篇高度匹配的文章。
Installation of 11.2.0.4 on RHEL 8.x failed with '/bin/ld: cannot find -lnsl make:Error in invoking target 'agent nmhs' of makefile ins_emagent.mk ' (Doc ID 2944859.1)
文章描述了是在RHEL 8.x上安装Oracle 11g遇到的问题。
其实在去年我也思考过类似的问题，并咨询过国产操作系统厂商kylin10和哪个操作系统及版本相似度高一些，比如redhat7、redhat8、centos、suse等，如果知道了kylin10和哪个大家熟知的系统相似度高了，很多问题就可以对比预测了，比如如果是和Centos7.5相似度高，就可以提前猜测到可能会遇到Oracle 11g在Centos7.5安装的那些问题。
厂商回复没有可比性，只是linux架构一样，其他都不太像。
这就只能走一步看一步了。
在回到问题，通过2944859.1文档可以看到：
在OL/RHEL 8.7及更高版本上安装11.2.0.4失败，在安装的relink阶段出现以下错误：
Error in invoking target 'links proc gen_pcscfg' of makefile '/precomp/lib/ins_precomp.mk'.Error in invoking target 'idg4odbc' of makefile '/rdbms/lib/ins_rdbms.mk'.Error in invoking target 'itnsping' of makefile '/network/lib/ins_net_client.mk'.Error in invoking target 'install' of makefile '/ctx/lib/ins_ctx.mk'.Error in invoking target 'nmo' of makefile '/sysman/lib/ins_emagent.mk'.Error in invoking target 'agent nmhs' of makefile '/sysman/lib/ins_emagent.mk'.Error in invoking target 'all_no_orcl' of makefile '/rdbms/lib/ins_rdbms.mk'.Error in invoking target 'install' of makefile '/network/lib/ins_net_client.mk'.Error in invoking target 'nmb' of makefile '/sysman/lib/ins_emagent.mk'.Error in invoking target 'install' of makefile '/sqlplus/lib/ins_sqlplus.mk'.Error in invoking target 'clientonlyinstall' of makefile '/ldap/lib/ins_ldap.mk'.Error in invoking target 'ioklist' of makefile '/network/lib/ins_nau.mk'.
也有文件为0的问题：
/bin> ls -l nm*-rws--x--- 1 root oracle 0 Aug 24 2013 nmb-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmb0-rwxr-xr-x 1 oracle oracle 12724 Jul 16 2013 nmcbufp-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmei-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmei0-rws--x--- 1 root oracle 0 Aug 24 2013 nmhs-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmhs0-rws--x--- 1 root oracle 0 Aug 24 2013 nmo-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmo0-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmocat-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmocat0-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmosudo-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmosudo0-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmupm-rwxr-xr-x 1 oracle oracle 0 Aug 24 2013 nmupm0
这个错误匹配度很高。
-
问题原因：
默认情况下，libnsl.so指向OEL v8主机上的libnsl.so.2，而EM Agent二进制文件的relink需要libnsl.so.1（特别是libnmemso.so共享库和emagent可执行文件）。
这是relink失败的原因，并根据未发布的Bug 35219138进行了分类:
Bug 35219138 - emagent relink during DB 11.2.0.4 installation fails with error "/bin/ld: cannot find -lnsl"
-
解决方案：
提供了两种解决方案：
方案一：
1) Perform software-only DB installation irrespective of errors observed during installation of software .2) Edit the file $ORACLE_HOME/lib/sysliblist ,ensure its modified as belowi,e cat $ORACLE_HOME/lib/sysliblist-ldl -lm -lpthread -l:libnsl.so.1 -lirc -lipgo -lsvml -laio3) Download and apply Merge Patch 35539202(merge of patches 19692824 and 35219138 )4) Relink oracle home binariescd $ORACLE_HOME/bin./relink all
需要更新Patch 35539202，考虑到Patch 35539202又不一定和银河麒麟系统兼容，所以最终没有使用这个方案。
方案二:
修改sysliblist,env_emagent.mk配置文件内容，重新生成config.o文件，重新执行relink，详细操作步骤如下：
a)Take backup and make changes to sysliblist filecd /lib/cp sysliblist sysliblist.bkpfrom-ldl -lm -lpthread -lnsl -lirc -lipgo -lsvml -laioto-ldl -lm -lpthread -l:libnsl.so.1 -lirc -lipgo -lsvml -laiob) Change the value of NMEHL_OSLIBS in/sysman/lib/env_emagent.mkfromNMEHL_OSLIBS=$(LDLIBFLAG)pthread$(LIB_SUFFIX) $(LDLIBFLAG)nsl$(LIB_SUFFIX) \$(LDLIBFLAG)m$(LIB_SUFFIX) $(LDLIBFLAG)dl$(LIB_SUFFIX)ToNMEHL_OSLIBS=$(LDLIBFLAG)pthread$(LIB_SUFFIX) $(LDLIBFLAG)m$(LIB_SUFFIX) \$(LDLIBFLAG)dl$(LIB_SUFFIX)c)fromMK_EMAGENT_NMECTL=$(LDCCOM) $(NMECTLMAIN) $(LIBNMECTL_LINK) $(LLIBCLNTSH) \$(NMEFET_STACKLIBS_LINK) $(USRLIBS) $(LDLIBS)ToMK_EMAGENT_NMECTL=$(LDCCOM) $(NMECTLMAIN) $(LIBNMECTL_LINK) $(LLIBCLNTSH) \$(NMEFET_STACKLIBS_LINK) $(USRLIBS) $(LDLIBS) $(LLIBNZ)d) Incase of if you notice $ORACLE_HOME/rdbms/lib/config.o as 0 bytes$ cd $ORACLE_HOME/rdbms/lib$ rm config.o$ make -f ins_rdbms.mk config.o ioracle$ ls -la config.oe) Re-link oracle binariescd $ORACLE_HOME/bin./relink all
其中，在执行 d 步骤时，检查config.o大小确实是0字节，但是在执行make操作时，报如下错误：
root@cjc-db-01:/xxxxx#make -f ins_rdbms.mk config.o ioracle/usr/bin/ld: cannot find /usr/lib64/libpthread_nonshared.acollect2: error: ld returned 1 exit statusmake: *** [ins_rdbms.mk:735: /db/oracle/app/oracle/product/11.2/db/rdbms/lib/oracle] Error 1
通过 libpthread_nonshared.a 关键字在MOS查到如下文章：
19c Grid Installation Failing For SLES15 SP4 with error "cannot find /usr/lib64/libpthread_nonshared.a" (Doc ID 2797420.1)
最终通过创建软连接解决此问题：
root@cjc-db-01:/root#find / -name libpthread_nonshared.a/usr/lib/libpthread_nonshared.a
添加软连接
ln -s /usr/lib/libpthread_nonshared.a /usr/lib64/libpthread_nonshared.a
再次执行，这次没有报错了：
root@cjc-db-01:/xxxxx#make -f ins_rdbms.mk config.o ioracle
继续执行 e 步骤：
oracle@cjc-db-01:$cd $ORACLE_HOME/binoracle@cjc-db-01:/db/oracle/app/oracle/product/11.2/db/bin$./relink allwriting relink log to: /db/oracle/app/oracle/product/11.2/db/install/relink.log
检查日志，没有异常：
/db/oracle/app/oracle/product/11.2/db/install/relink.log
检查之前为0字节的二进制文件已恢复正常，lsnrctl等工具可以正常使用。
oracle@cjc-db-01:/db/oracle/app/oracle/product/11.2/db/bin$ls -lrth lsnrctl-rwxr-x--x 1 oracle oinstall 156K Apr 3 12:51 lsnrctloracle@cjc-db-01:/db/oracle/app/oracle/product/11.2/db/bin$ls -lrth rman-rwxr-x--x 1 oracle oinstall 17M Apr 3 12:51 rman......
后面的静默执行dbca等操作没有遇到问题。
参考：
Installation of 11.2.0.4 on RHEL 8.x failed with '/bin/ld: cannot find -lnsl make:Error in invoking target 'agent nmhs' of makefile ins_emagent.mk ' (Doc ID 2944859.1)19c Grid Installation Failing For SLES15 SP4 with error "cannot find /usr/lib64/libpthread_nonshared.a" (Doc ID 2797420.1)
###chenjuchao 20240403###
