NLS_LANG, character sets & more – Part 2

Some days ago I wrote about NLS_LANG, character sets & more. Today I saw another neat effect. I was trying to do it right, but my messages from SQL*Plus still looked scrambled:


The thing is, that the characters I need are simply not present in the default font. When I change the font, the same screen now looks somehow different:


Maybe I should switch from German to English and ASCII 7 completely…

NLS_LANG, character sets & more – How to do it right


Globalization is confusing. Everyone and everything is using different characters in different charcter sets. Oracle is prepared for that and offers several parameters and variables to control the behaviour. But one must be careful when setting these, that’s why I want to give a rough overview of the basic things.


When sending or retrieving character data to/from a database, there are three to four settings that can influence the display of that data.


  1. That is the encoding inside the database itself.
  2. The character set which the Oracle Client is using to display character data.
  3. The charcter set that is used by the operating system at the client side.
  4. The character set that PuTTY (or whatever terminal emulation you prefer) asumes the remote side is using.

The picture outlines the route which character data takes during the process of reading from or writing to the database. In this post I will talk about the yellow part of the diagram.

Let’s start with the database. There is some character data stored inside that is encoded using the configured database characterset. Now we want to retrieve data from the database. That means the Oracle Client (2) sends the SQL to the database (1) and in turn gets the data which is being converted automatically to the character set that is configured for the Oracle Client. That is typically done using the NLS_LANG variable. Beside that, this is the only point where a character set conversion might happen.

Next the charcter data is being displayed by the operating system (3) which is using it’s own character set. There is no conversion anymore, that means our NLS_LANG setting must match the setting of our OS.

And lastly maybe there is a terminal emulation like PuTTY which also defines a character set which obviously must match to the one used by the OS we are connecting to.

What can go wrong

Having this process in mind we see, that wrong parameter settings may not be recognized immediately. When inserting data into the database with wrong settings and then querying this data will get correct results as the translation is done in the same way. We start seeing wrong characters when we query the data with the correct settings.

Let’s say we have a Windows client and use SQL*Plus inside CMD to insert data. The system wide NLS_LANG variable is set to MSWIN1252 as Windows is using this character set. But as described in a previous post the CMD is using another character set, PC850.

So let’s create a table, insert some data and query that data:


So we see that the special characters that I inserted are displayed correctly when querying the data because the same wrong transformation happens in both directions. More or less, I have no idea why the Euro sign gets messed up… Maybe because PC850 has no Euro sign…

Now I query the data from SQL Developer which is using the windows character set to display data:


Now again a transformation might take place, depends on the database setting. And this time we see wrong data because the transformation was wrong when I inserted the data.

Next step, insert data with SQL Developer:


Inserted and displayed correctly. But obviously CMD shows it as follows:


This is all simply because the CMD renders characters in a different character set as the rest of Windows. So when we change CMD to use the proper code page, it looks like this:


Now the first dataset is rendered differently, but the data from SQL Developer is shown properly.

Another way to display the data properly is to modify the NLS_LANG setting inside CMD:


Only the Euro sign is missing since it is not part of the PC850 character set.

Now I set the NLS_LANG again to MSWIN1252 and insert a third record:


Looks good so far, but again, I should crosscheck that with SQL Developer:


Ok, the data is still displayed properly. So this is the correct setting that we should use for Windows.

But what about Linux? Linux is using UTF8 internally:


So I should set NLS_LANG to AL32UTF in order to get my data displayed correctly:


As expected, the data shows up as it should be. But this is only because my PuTTY is using the right setting. What if I modify PuTTY to use MSWIN1252? Might be a valid setting because my Windows where PuTTY runs on is using that character set:


How does the result now looks like?


Totally messed up since my multibyte output from Linux is being interpreted as singlebyte. So that is not a good idea. The PuTTY character set setting must match the character set that is used by the OS that we connect to:



Be careful when setting NLS parameters on both, client and server, side. You might not notice a misconfiguration as long as you are using the same track for data retrival and insertion. Just use another client to crosscheck the data that you are dealing with. It all depends on the OS and the correct NLS_LANG setting at the client side.

There is a good FAQ from Oracle that outlines the whole NLS things.

Default Collation and PL/SQL

One of the new features that got introduced in Oracle Database 12.2 is the possibility to define the default collation at schema-, table- or column-level. This is quite nice because we do not have to use NLS_SORT in each and every query anymore.
But obviously there are some restrictions. Let’s create two users, one with no specific collation and one that uses a non-default collation.

SQL> show user
SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         3 PDBMMI                         READ WRITE NO

SQL> create user usr_default identified by xxx;

User created.

SQL> grant create session, alter session, create procedure to usr_default;

Grant succeeded.

SQL> create user usr_collate identified by xxx default collation binary;

User created.

SQL> grant create session, alter session, create procedure to usr_collate;

Grant succeeded.

Now I create a simple piece of PL/SQL, first with the default user.

SQL> conn usr_default/xxx@pdbmmi
SQL> show user
SQL> select default_collation from user_users;


SQL> create procedure do_nothing
  2  as
  3  begin
  4    null;
  5  end;
  6  /

Procedure created.

SQL> exec do_nothing;

PL/SQL procedure successfully completed.

SQL> drop procedure do_nothing;

Procedure dropped.

Nothing special here. Everything works as expected and as we know it since years. So let’s try the same with the user that has a different default collation.

SQL> conn usr_collate/xxx@pdbmmi
SQL> show user
SQL> select default_collation from user_users;


SQL> create procedure do_nothing
  2  as
  3  begin
  4    null;
  5  end;
  6  /

Warning: Procedure created with compilation errors.

SQL> show err

-------- -----------------------------------------------------------------
0/0      PL/SQL: Compilation unit analysis terminated
0/0      PLS-00761: Program unit collation may only be USING_NLS_COMP

Obviously the PL/SQL compiler is not very happy with the different collation option. So we need to change that, at session level in my case, to make it compile.

SQL> alter session set default_collation=USING_NLS_COMP;

Session altered.

SQL> alter procedure do_nothing compile;

Procedure altered.

SQL> show err
No errors.
SQL> exec do_nothing;

PL/SQL procedure successfully completed.

SQL> drop procedure do_nothing;

Procedure dropped.

Now it compiles without any warnings or errors.

So keep that in mind when you create users with a specific collation option.

Oracle 12.2 – SQL*Plus Command History

Currently I am preparing some slides for future events. I will share some of the topics beforehand. First topic is the one mentioned in the title, with Oracle Database 12.2 we finally get a command history. Though only a tiny new bit this is one of my favorite new features.
Let’s see how this works.

$ sqlplus / as sysdba

SQL*Plus: Release Production on Mon May 8 09:49:04 2017

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Connected to:
Oracle Database 12c Enterprise Edition Release - 64bit Production

SQL> history
SP2-1650: History is off, use "SET HIST[ORY] ON" to enable History.

Ok, the feature is inactive by default. We need to enable it by setting it to “ON” or to a number specifying how many statements should be kept in the history.

SQL> set history 50

SQL> select * from dual;
SQL> history
  1  select * from dual;

Let’s see what else is possible with the “history” command:

SQL> help history


 Stores, lists, executes, edits of the commands
 entered during the current SQL*Plus session.


 N is the entry number listed in the history list.
 Use this number to recall, edit or delete the command.

 HIST 3 RUN - will run the 3rd entry from the list.

 HIST[ORY] without any option will list all entries in the list.

We can easily run any command by specifying the number from the history or remove a command from the history.

SQL> hist
  1  select * from dual;
  2  select count(*) from dba_objects where status <> 'VALID';

SQL> hist 2 run


SQL> hist 1 del
SQL> hist
  1  select count(*) from dba_objects where status <> 'VALID';

And by the way, the history runs at statement level, not at line level. So this is maybe the most important advantage over OS level command history using arrow-up and arrow-down (natively on Windows or with “rlwrap” on Linux).

SQL> select count(*) from dba_users
  2  where oracle_maintained = 'Y'
  3  and account_status = 'OPEN';


SQL> hist
  1  select count(*) from dba_objects where status <> 'VALID';
  2  select count(*) from dba_users
     where oracle_maintained = 'Y'
     and account_status = 'OPEN';

SQL> hist 2 run


Enjoy the new feature.

ORA-28545 when querying HS gateway in Oracle 12c

Since our customers are forced to move to Oracle Database 12.1, I am in the role to support their upgrades. Recently I upgraded a Windows environment that used the Heterogeneous Services to connect to an external ODBC datasource via a database link. The configuration is a little tricky, but pretty straight forward once you get the idea.

First, there must be an ODBC datasource. That is a simple System DSN which was already in place.

Second, you have to choose a kind of “virtual” SID for your connection to this ODBC datasource.

Third, create an init.ora in the $ORACLE_HOME/hs/admin directory and put at least one parameter into it:


Fourth, add static SID to the listener  by modifying the listener.ora file:

(SID_NAME=<virtual SID>)

Then reload the listener configuration to make it active.

Fifth, create a tnsnames.ora entry that points to the listener and the virtual SID. Make sure, you specify “HS=OK”.

(ADDRESS = (PROTOCOL = tcp)(HOST = <listener host>)(PORT = 1521))
(SID = <virtual SID>)
(HS = OK)

Sixth and finally, create a database link that points to the tnsnames entry:

SQL> create database link myodbcds connect to <user> identified by <pwd> using "MYODBCDS";

Database link created.

All those steps were done years ago in the old 11.2 environment. Everything worked fine there. So I simply copied all the configuration details to the new 12.1 Oracle home. Then I moved the listener to the new 12.1 home and upgraded the database. But when testing the HS connection, the following happened:

SQL*Plus: Release Production on Tue Mar 28 11:30:45 2017

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Last Successful login time: Tue Mar 28 2017 11:06:02 +02:00

Connected to:
Oracle Database 12c Standard Edition Release - 64bit Production

SQL> select * from dual@myodbcds ;
select * from dual@myodbcds
ERROR at line 1:
ORA-28545: error diagnosed by Net8 when connecting to an agent
Unable to retrieve text of NETWORK/NCR message 65535
ORA-02063: preceding 2 lines from MYODBCDS

I checked the listener services, the connection seemed to be established successfully.

C:\>lsnrctl services listener

LSNRCTL for 64-bit Windows: Version - Production on 28-MAR-2017 11:39:05

Copyright (c) 1991, 2014, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=<hostname>)(PORT=1521)))
Services Summary...
Service "<virtual SID>" has 1 instance(s).
Instance "<virtual SID>", status UNKNOWN, has 1 handler(s) for this service...
"DEDICATED" established:1 refused:0
The command completed successfully

So I created a new listener in the old 11.2 home that used another port to test things. The 11.2 listener worked fine. So what is the difference between both listeners?
After a little research I stumbled upon the Windows services. The database service was not running as LocalSystem but as a domain user. This is because the database writes it’s backups to an UNC share. Both, 11.2 and 12.1, listeners used the LocalSystem account. So I changed the 12.1 listener to use the domain account too and this solved the problem.

SQL*Plus: Release Production on Tue Mar 28 11:39:10 2017

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Last Successful login time: Tue Mar 28 2017 11:36:53 +02:00

Connected to:
Oracle Database 12c Standard Edition Release - 64bit Production

SQL> select * from dual@myodbcds;


I have no idea why the 11.2 listener can run as LocalSystem whilst the 12.1 listener cannot. But generally I think it is a good practice to run listener and database services with the same user credentials.

12c Upgrade stuck at fixed object stats

A while ago I did some database upgrades to version One of those upgrades made me a bit nervous. I used “dbca” to perform the upgrade and it was kind of stuck. So I went to the command prompt and started to investigate this. What I found was this:


A session was waiting for “ADR block file read” again and again. The SQL that was causing this waits can be seen in the screenshot too. It was gathering statistics for X$DBKFDG, part of the “gather fixed object stats” step.

A quick research in My Oracle Support did not bring up any helpful information. My first idea was to check the ADR but I found it to be nearly empty, at least in the suspicous directories like “trace” etc. So I posted this issue on Twitter and asked for help. @realadrienne pointed to a similar issue related to Recovery Advisor and the list of critical issues. So I checked that using

SQL> select * from v$ir_failure;

which returned loads of records that had a priority of “critical” and a status of “closed”. So I tried to get rid of these records using “adrci”.

adrci> purge -age 1 -type hm

This took quite a while, about 10 minutes. to complete. But in the end the v$ir_failure view had no records anymore and the fixed object stats where gathered quickly.

So now there is an additional pre-upgrade check on my personal list, that says “clean up v$ir_failure”. You should add this to your list too to prevent unnecessary delays during database upgrades.

Thanks to @realadrienne, @FranckPachot and @OracleSK for the quick and very helpful assistance.

Thanks also to @MikeDietrichDE for the feedback on this issue.



Things to consider when moving a database

A main task of a DBA’s life is keeping database versions and hardware up-to-date. This sounds easy, but actually it is not. This is especially if the system was not touched for a long period of time. And there might also be a lot of applications using a database. This implies dependencies.
The todays story is about a server migration that we did a couple of weeks ago. It meant moving 7 databases to a new hardware which we did by simply stopping the databases on the old server, stopping ASM on the old server, mounting the ASM disks to the new server(s) and creating cluster resources for the databases. But there were several other things to take care of. The following list is long but might not be complete, so please feel free to comment on that to make the list as good as possible.

  • cron jobs of ALL users
    • copy cron entries and all references scripts/directories etc
  • scripts
    • copy any scripts that are on the old system
  • local applications
    • copy any application and their configuration that might run on the old system
  • tnsnames.ora of ALL clients
    • prepare TNS aliases for the new system and activate them during the migration
    • notify all application owners who do not use tnsnames.ora or directory services (JDBC Thin, EZ-Connect, …)
  • external procedures
    • be ea
  • directories
    • adump
    • diagnostic_dest
    • directory objects
  • database links in ALL other databases
  • backup
    • RMAN configuration, especially locations for backups and snapshot controlfile
  • connect identifiers in applications, that do not use tnsnames.ora (JDBC etc)
  • SSH keys, authorized_keys
    • prepare any remote systems that were able to login via SSH to work seamlessly with the new server
    • add any neccessary entries to known_hosts and/or authorized_keys on the new server
  • Monitoring systems
    • change your central monitoring tool to monitor the new environment


Once you moved everything to the new system it might be a good idea to keep the Oracle Listener(s) up and running on the old system for a while. Doing that, you will be able to monitor the listener logfiles and identify any application/user/interface that were not updated and  are still trying to connect to the old environment.