IQ Main store cannot be larger than licensed limit
[SAP IQ] IQ doesn't support WITH/WITH RECURSIVE clause.
Dear all,
I have many customers who are getting below error when they are using with or with recursive clause.
[Error]
Unable to execute the statement.
Remote server does not have the ability to support this statement.
SQLCODE=-706, ODBC=3, STATE="HY000" Row 30, Column 1
IQ does not support with/with recursive clause so far.
WITH clause is introduced in SQL Anywhere documentation only.
...
SELECT statement
Use this statement to retrieve information from the database.
Syntax
[ WITH temporary-views ]
SELECT [ ALL | DISTINCT ] [ row-limitation ] select-list
[ INTO { hostvar-list | variable-list | table-name } ]
NFR(New Feature Request) is in progress as follows. But It's not included in IQ yet.
"NFR#607124 - IQ does not include ANSI SQL-99 select .. with recursive queries."
Thanks
HTH
Gi-Sung Jang
Blocks in sybase iq
Anyone explain me the difference between Physical Blocks, NBlocks and Catalog Blocks in Sybase iq.
Thanks,
Santosh Negalur
Sizing of Large Memory cache in IQ 16
Hi
I'm in the middle of a migration project from IQ 12.7 to IQ 16 SP4.5. Currently, 96GB is allocated for IQ on the new RHEL server and I have configured Main Cache, Temp Cache and Large Memory to 32GB each. However, I have problems with loading of relatively wide tables. When loads into two wide tables run concurrently, one of LOADs fail and I receive following messages in IQ Message Log:
I. 11/04 12:23:29. 0000000070 Exception Thrown from slib/s_llhtable.cxx:187, Err# 13, tid 2 origtid 2
I. 11/04 12:23:29. 0000000070 O/S Err#: 0, ErrID: 517 (hos_memexception); SQLCode: -1013130, SQLState: 'QBC75', Severity: 14
I. 11/04 12:23:29. 0000000070 [22160]: All IQ large memory has been used, allocation canceled [size: 33554495]
-- (slib/s_llhtable.cxx 187)
The size "33554495" is always the same.
When I increase -iqlm to 48GB the problem doesn't occur. Monitoring via sp_iqsysmon (-section lma) reveals that when run separately, the first load allocates about 33GB of "Large Memory Inflexible" and the second load allocates about 10GB.
When run concurrently, the consumption of "Large Memory Inflexible" is about 42 GB. The first table has 766 columns and the second one 572, mostly NBit FP indexes.The number of rows inserted in each load is not big at all, fewer than a million. There are no conversions to flat FP during the loads.I use default values for FP_NBIT_Autosize_Limit and FP_NBIT_Lookup_MB.
Questions:
1. Do such massive memory requirements make sense at all? Does it sound like a normal behavior or a bug?
2. I read the section about the large memory cache in the IQ Sizing Guide. Still, it is not clear to me how to calculate the requirements for loads of NBit indexes. Specifically, is there a way to calculate it precisely based on the output of sp_iqindexmetadata (TokenCount, CountSize, DictSize)?
3. In IQ 15.*, FP(3) indexes proved to be problematic. In many cases, FP(3) brought more harm than good, in some cases I had to just ban using of such indexes entirely to avoid a significant drop in performance. Now we have NBit indexes, which potentially can grow even beyond the limits of FP(3). Specifically, I have tens of NBit indexes with >16 bits on the first table in my test case. Should it be considered a good practice? Maybe it worth converting such indexes to FlatFP, geven that I prefer better performance over storage savings?
Thanks in advance
Leonid Gvirtz
How check when was a table last accessed?
We can get information about DML changes on a table with sp_iqunusedtable. However, our requirement needs when was a Select query was executed on a particular table, it is possible??
Best regards
com.sybase.scc.jmx.ManagedObjectGateway.err_invoke_op SCC agent db2v105:9999 does not know Sybase IQ version string. Something is wrong with the IQ Agent plug-in.
Hello All,
I am a student and just want to try my hands on SAP IQ. So i downloaded SAP for linux. I set up a Virtual machine on my windows and loaded the linux image onto the virtual machine. So far i have been successfully able to install SAP IQ, create database and so on. But I have only got as far as authenticating the agent after registering it. When i authenticate the agent it gives me the below error and obviously the authentication fails
"com.sybase.scc.jmx.ManagedObjectGateway.err_invoke_op SCC agent db2v105:9999 does not know Sybase IQ version string. Something is wrong with the IQ Agent plug-in."
So i searched on internet and followed the process mentioned in SAP info center under change request number 723112 however I am still unable to get it working. I would be really grateful if someone could help me ASAP. Thanks in advance
SQL Anywhere Error - 210: User 'another user' has the row in 'tablexyz' locked
Hi I have multiple Java process which launches serially one by one. This process does job of loading data into tablexyz and then tablefinal. After first process finishes its job I get the following error
SQL Anywhere Error - 210: User 'another user' has the row in 'tablexyz' locked.
Flow goes like this:
Process 1:
Creates table tablexyz and loads data into it does some processing and then finally copies data into tablefinal. I commit transaction here after all steps.
Process 2:
Drops/Empties table tablexyz if it is there and then againcreates table tablexyz and loads data into it does some processing and then finally copies data into tablefinal. I commit transaction here after all steps.
My app fails at the process 2 and it gives tablexyz locked. I dont understand both process uses same database user myuser then why second process is not able to delete tablexyz created by first process. Please guide I am new to Sybase IQ. Thanks in advance.
Date conversion behavior in IQ
- IQ converts a date value '00000101' to '2001-01-01', while it does a date value '0099-01-01' to '1999-01-01' unexpectedly.
- This commonly happens in SAP SYBASE IQ 15.2, 15.3, 15.4, 16.0.
- The steps for reproducing this are as follows.
- Create a table which contains a DATE data type.
- create table TAB_1(converted_dt date) ;
- Load data to the table
- Flat file
- Create a table which contains a DATE data type.
- Load table statement
- Execute the query to see the converted data.
- A date value '00010101', 00990101 were converted to '2001-01-01', '1999-01-01' repectively.
- Cause
- Resolution
- Specify a date format in the column spec of the load table statement.
load table TAB_11(converted_dt date('yyyymmdd'), filler('\n'))
from '/home2/iq16/WORK/test.txt'
quotes off
escapes off;
commit;
=>
converted_dt
---------------
0001-01-01
0010-01-01
0099-01-01
0101-01-01
1000-01-01
1989-01-01
- Make a date value with seperator of year, month and day, for example '0001-01-01' instead of '00010101'.
- You can see this information in manual pages below.
[IQ 16.0]
http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc00801.1604/doc/html/san1281565160724.html
http://help.sap.com/saphelp_iq1608_iqrefso/helpdata/en/a6/42956484f21015a7a0e61656a45f16/content.htm?frameset=/en/a8/90e66f84f210158e8f869393336af4/frameset.htm¤t_toc=/en/a8/90e66f84f210158e8f869393336af4/plain.htm&node_id=379
HTH
Jerry
Load table problem - Non-space text found after ending quote character for an enclosed field
Hi folks,
I am getting the error "Non-space text found after ending quote character for an enclosed field" when I attempt to load data into a table using the statement below. Any ideas how to resolve this issue?
load table FACTS_29_4d29401c_b16e_4914_adfd_2f5ca94bbe7b(id_1 null(ZEROS), id_2 null(ZEROS), id_3 null(ZEROS), id_4 null(ZEROS), id_5 null(ZEROS), id_6 null(ZEROS), id_7 null(ZEROS), id_8 null(ZEROS), id_9 null(ZEROS), id_10 null(ZEROS), id_11 null(ZEROS), id_12 null(ZEROS), id_13 null(ZEROS), id_14 null(ZEROS), id_15 null(ZEROS), id_16 null(ZEROS), id_17 null(ZEROS), id_52 null(ZEROS))
from 'C:\\test2.csv' escapes off
The file I am loading is attached (test2.csv). It contains the following data:
'1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','52'
Note, there is a new line after the '52' (so the file contains two lines - the one above and a blank line). The load works when there is no new line after the '52'.
The schema of the table I am loading it into is:
Column,Type,Nullable,Primary Key
'id_1','integer',1,0
'id_2','varchar(14)',1,0
'id_3','varchar(10)',1,0
'id_4','varchar(11)',1,0
'id_5','varchar(5)',1,0
'id_6','integer',1,0
'id_7','varchar(5)',1,0
'id_8','varchar(5)',1,0
'id_9','varchar(11)',1,0
'id_10','integer',1,0
'id_11','varchar(5)',1,0
'id_12','integer',1,0
'id_13','integer',1,0
'id_14','integer',1,0
'id_15','integer',1,0
'id_16','varchar(9)',1,0
'id_17','varchar(5)',1,0
'id_52','integer',1,0
Cheers,
Máirtín
Sybase IQ 16 startup issues
Sybase IQ 16 on windows
I have installed SAP IQ 16 Demo system and created Database based on RDS documents .
I have started and stopped server several times , it was working . there is not much activity on this server after installation.
When I tried to start today its not starting due to some Virtual Memory issues.
Error below :
Starting Sybase IQ > All available virtual memory has been used; allocation cancelled+info: 67112959
Database server shutdown due to startup error.
Pls advise me if you have any inputs to resolve this issues.
6/01 04:16:13. All available virtual memory has been used; allocation cancelled: [Extra info: 67112959]
-- (oslib\hos_mem.cxx 573)
06/01 05:13:38. All available virtual memory has been used; allocation cancelled: [Extra info: 67112959]
-- (oslib\hos_mem.cxx 573)
06/03 16:49:29. All available virtual memory has been used; allocation cancelled: [Extra info: 67112959]
-- (oslib\hos_mem.cxx 573)
09/02 03:14:04. No SYSAM License Available
11/11 13:17:22. All available virtual memory has been used; allocation cancelled: [Extra info: 67112959]
-- (oslib\hos_mem.cxx 573)
11/11 14:14:35. All available virtual memory has been used; allocation cancelled: [Extra info: 67112959]
-- (oslib\hos_mem.cxx 573)
11/11 16:17:25. All available virtual memory has been used; allocation cancelled: [Extra info: 67112959]
-- (oslib\hos_mem.cxx 573)
Multiplex and File system
Hi All
What is the future of using multiplex with file system and direct I/O. In the Linux world file system with direct I/O seems to be the norm and simplex can be run this way. What is the stumbling blocks in using shared file system with Multiplex?
regards
Johan Bornman
error while reading the table data
Hi,
I extracted the table data to the file .csv , after when I tried to query that table its giving the above error.
the output of sp_iqlocks
I am not able to read the table data after giving commit to unlock the lock.
In which isolation level the Sybase iq works? How to check the isolation level it is working?
Please any suggestion.......
[SAP IQ] Silent Crash of IQ.
Problem
IQ sometimes gets aborted silently logs on Linux.
In this case there are no stacktraces and error messages.
Cause
An OOM Killer (Out of memory Killer) kills a process when the no available free memory on system.
If IQ process is killed by OOM Killer, IQ will crash without any error and stacktrace.
We can see the "out of memory" within message log.
[message log]
Nov 12 12:48:00 BDDBBETLMV01 kernel: iqsrv16 invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Nov 12 12:48:00 BDDBBETLMV01 kernel: iqsrv16 cpuset=/ mems_allowed=0
Nov 12 12:48:00 BDDBBETLMV01 kernel: Pid: 29293, comm: iqsrv16 Not tainted 2.6.32-279.el6.x86_64 #1
Nov 12 12:48:00 BDDBBETLMV01 kernel: Call Trace:
Nov 12 12:48:00 BDDBBETLMV01 kernel: [<ffffffff810c4971>] ? cpuset_print_task_mems_allowed+0x91/0xb0
Nov 12 12:48:00 BDDBBETLMV01 kernel: [<ffffffff811170e0>] ? dump_header+0x90/0x1b0
Nov 12 12:48:00 BDDBBETLMV01 kernel: [<ffffffff812146fc>] ? security_real_capable_noaudit+0x3c/0x70
Nov 12 12:48:00 BDDBBETLMV01 kernel: [<ffffffff81117562>] ? oom_kill_process+0x82/0x2a0
Nov 12 12:48:00 BDDBBETLMV01 kernel: [<ffffffff8111745e>] ? select_bad_process+0x9e/0x120
Nov 12 12:48:00 BDDBBETLMV01 kernel: [<ffffffff811179a0>] ? out_of_memory+0x220/0x3c0
...
...
Nov 12 12:48:00 BDDBBETLMV01 kernel: Out of memory: Kill process 29081 (iqsrv16) score 950 or sacrifice child
Nov 12 12:48:00 BDDBBETLMV01 kernel: Killed process 29081, UID 501, (iqsrv16) total-vm:24941028kB, anon-rss:3586296kB, file-rss:1332kB
Resolution
It's not an IQ issue, It's resource issue.
There are a couple of solutions on it.
1) Increase the physical memory or decrease the value of iqmc/iqtc/-ch/-cl
2) Decrease the number of concurrent jobs
HTH
Regards
Gi-Sung Jang
Sybase IQ 15.4 Client side load on Linux using JDBC iAnywhere driver
Hi I am trying to do client side load on Linux and getting the following error:
Exceptioninthread"main"java.lang.UnsatisfiedLinkError: /home/local/libdbdb12_r.so:libdbtasks12_r.so:cannotopensharedobjectfile:Nosuchfileordirectory
I am using iAnywhere driver as shown below
Class.forName("ianywhere.ml.jdbcodbc.jdbc3.IDriver") //to load this driver I am using jodbc.jar which comes under IQ15 client java folder
My jdbc url is given below:
String jdbcURL = "jdbc:ianywhere:driver=libdbodbc_r.so;UID=user;PWD=pass;DBN=sMDNC;eng=IQSR1;" +
"CommLinks=tcpip{host=xyz.com;port=2345}";
I am also setting LD_LIBRARY path to /home/local where libdbodb_r.so file is residing but still getting error. Please help I am new to Sybase IQ.
[License] How to solve the problem of license mismatch.
1. Problem
IQ sometimes fails to obtain license due to mismatch of license type between license file and servername.lmp file.
[Error]
I. 11/04 17:21:11. 0000000000 Using licenses from: /Sybase/SybaseIQ16/Binary/sybase/SYSAM-2_0/licenses/Core_License.lic.lic:/Sybase/SybaseIQ16/Binary/sybase/SYSAM-2_0/licenses/BDDBBETLMV01_20141024091447.lic:/Sybase/SybaseIQ16/Binary/sybase/SYSAM-2_0/licenses/BDDBBETLMV01_20141024091957.lic:/Sybase/SybaseIQ16/Binary/sybase/SYSAM-2_0/licenses/SYBASE.lic:/Sybase/SybaseIQ16/CATALOG_STORE/DWDEV/*.lic
I. 11/04 17:21:12. 0000000000 Checked out graced license for 4 IQ_CORE (2019.09300/permanent/0244 9940 B657 35CC) will expire Fri 21 Nov 2014 12:41:45 PM IST.
I. 11/04 17:21:12. 0000000000 Failed to obtain license(s) for IQ_CORE feature from license file(s) or server(s).
2. Solution
In most cases,
1) This problem can be solved by changing LT(License Type) within servername.lmp to same value of license file.
2) Or Removing the lmp file. In this case, When IQ start-ups, servername.lmp file is generated.
If the LT/PE value has not been changed Even though changing or removing the lmp file.
3) You have to look at the iq.default.lmp file as follows.
~/IQ-16_0/Sysam/iq.default.lmp
If the LT/PE value within iq.default.lmp is different from xxx.lmp or license file,
You have to change the LT value to same value of license file or remove the iq.default.lmp file.
Because the xxx.lmp will use PE/LT values given in iq.default.lmp.
**The value of PE/LT within iq.default.lmp is decided when Installation of IQ.
HTH
Thanks
Gi-Sung Jang
Space management issues
Hello. I am completely new to Sybase IQ. I am trying to compare the performance of a stored procedure on SQL and Sybase IQ. I am using Sybase IQ version 16. I created a database and some tables and loaded the tables with data ( very large tables, records ranging from 10-40 million) . When I execute a stored procedure in this database , errors like
1) Could not execute statement.
main Bufman: All buffer cache pages are in use, ask your DBA to increase
the size of the buffer cache.
-- (slib\s_blockmap.cxx 12085)
SQLCODE=-1009031, ODBC 3 State="HY000"
Line 1, column 1
exec "DBA"."usp_create_det_f_create_so_to_del_to_inv"
and
2) IQ_System_temp is full.
How could I solve this issue and execute the stored procedure
Does exist any procedure to identify a table partition size?
I have some partitioned tables, and I need to identify the size of each partition, it is possible??
Best regards,
JC
Export and Import
Hi experts,
1. I use the dbisql to export a table to a file
(DBA) > SELECT * FROM Employees ># /sybase/empfile.csv
How I can specify the codepage (eg: 1208) for the export?
2. I want to import the above file to the table again.
I have studied the "LOAD TABLE" statement, but it has too many parameters. I feel difficult to find the correct
parameters to load the file.
For example, I use the statement below:
LOAD TABLE Employees (EmployeeID,ManagerID,Surname,GivenName,DepartmentID,Street,City,State,Country,PostalCode,Phone,Status,SocialSecurityNumber,Salary,StartDate, TerminationDate,BirthDate,BenefitHealthInsurance,BenefitLifeInsurance,BenefitDayCare,***) from '/sybase/empfile.csv' ESCAPES OFF FORMAT ascii DELIMITED BY ',' |
I did not modify the file after exporting the file.
But it returns the "non-space text found after ending quote character for an enclosed field" errors.
It seems Sybase IQ does not take the "import" and "export" as a pair of actions.
It is difficult to import the file to a table after exporting the data to a file. Strange.
Thanks & Regards,
Gordon
Fast data fetch
Hi, what is the most efficient way to fetch data into memory from Sybase IQ database with C++?
Is using SAP IQ Database API for C/C++ much faster than using ODBC, or there is no big difference between this two APIs?
Thank you
IQ REPLACE LOAD
Hello Guys,
I want to load from CSV file to IQ16 table , but this table has primary key and CSV has existing primary key data.
I want to take this existing data as update , new data as insert in IQ Load command. But It seems to me, IQ Loader doesn't have this feature .
I am thinking another method ... ie) Load to another table and use insert-select command and update command .
However , I think some guys has same trouble in IQ. How to solve this case ? Does anyone know good method ?
And I googled about this, "Sybase ETL" has this feature for IQ. But this products is obsoleted? Data Services has same feature ?
Rgs,
Jim