Question: I have been using a piece of software called FDC that interefaces with CSI’s API to access the data for neary four years now with out any problems. I recently purchased a new laptop with the Windows 7 Ultimate OS (64 bit) on it. On this computer the FDC software is not able to successfully interface with the API. What could be wrong?
Answer: The following keys should be present in your registry, if not, then something is preventing the UAd.exe from registering itself when it is run. Please make sure you run UA from an admin account at least once for the entries to be entered.
HKEY_CLASSES_ROOT\TypeLib\{6F2F99A0-6600-11D3-B331-525400E8C92C}\1.0\0\win32\”C:\ua\uad.exe”
HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{6F2F99A3-6600-11D3-B331-525400E8C92C}\LocalServer32\”C:\ua\uad.exe /Automation”
HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{A7120BAF-D87E-4949-93A0-53BB63F4D528}\LocalServer32\”C:\ua\uad.exe /Automation”
HKEY_CLASSES_ROOT\Wow6432Node\TypeLib\{6F2F99A0-6600-11D3-B331-525400E8C92C}\1.0\0\win32\”C:\ua\uad.exe”
Question: Why does the UAD.exe program hang when it is called from a perl script for the third time while running. I’m running the uad.exe program on a windows server 2003 machine with all patches.
Answer: Try this first, prior to running from perl, ensure that the file “windows\ua.ini” is set to the correct directory, like:
[Program]
Directory=C:\ua\
using SetPrivateProfileString
Try setting your scripts working directory as C:\UA
Question: Moving from successful Excel API interface to to VS2010. Windows 7 x64 with VS2010 Ultimate. Can you offer a code sample for the Current release of Visual Studio?
Answer: Will you be using C++ or C#?
If using C++ here is a sample array accessor to use after a request: ( you’ll need to import the com class from ua\uapi\ua.tlb )
IAPI2 *UAapi = new IAPI2();
if(UAapi==NULL)
{
AfxMessageBox(“Unable to open connection to UA stub object”);
return;
}
if(!UAapi->CreateDispatch(“UA.Api2”))
{
AfxMessageBox(“Unable to open connection to UA”);
return;
}
UAapi->SetApplyCommodityAdjustments(0);
UAapi->SetRoundToTick(0);
VARIANT IArray, DArray;
VariantInit(&IArray);
VariantInit(&DArray);
count= UAapi->CopyRetrievedDataToArray2(0, &IArray , &DArray);
char strLineOut[128];
long NumIntColumns, NumFltColumns;
{ SafeArrayGetUBound(IArray.parray,1,&NumIntColumns); NumIntColumns++;
SafeArrayGetUBound(DArray.parray,1,&NumFltColumns); NumFltColumns++;
}
long *iArray; SafeArrayAccessData(IArray.parray,(void**)&iArray);
double *dArray; SafeArrayAccessData(DArray.parray,(void**)&dArray);
for(int c=0; c < count; c++ )
{
#define sGetDate(c) iArray[NumIntColumns*(c)+0]
#define sGetDyDm(c) iArray[NumIntColumns*(c)+1]
#define sGetStrike(c) iArray[NumIntColumns*(c)+2]
#define sGetVol(c) iArray[NumIntColumns*(c)+3]
#define sGetTVol(c) iArray[NumIntColumns*(c)+5]
#define sGetOi(c) iArray[NumIntColumns*(c)+4]
#define sGetTOi(c) iArray[NumIntColumns*(c)+6]
#define sGetfStrike(c)dArray[NumFltColumns*(c)+0]
#define sGetOpen(c) dArray[NumFltColumns*(c)+1]
#define sGetHigh(c) dArray[NumFltColumns*(c)+2]
#define sGetLow(c) dArray[NumFltColumns*(c)+3]
#define sGetClose(c) dArray[NumFltColumns*(c)+4]
#define sGetBid(c) dArray[NumFltColumns*(c)+5]
#define sGetAsk(c) dArray[NumFltColumns*(c)+6]
// print to file the commod code
{ sprintf(strLineOut, “%d,%d,%2.8f00, %2.8f00, %2.8f00, %2.8f00, %d,%d\n”, sGetDate(c),
sGetDyDm(c),
sGetOpen(c),
sGetHigh(c),
sGetLow(c),
sGetClose(c),
sGetVol(c),
sGetOi(c)
fputs(strLineOut, ptr);
}
}
SafeArrayUnaccessData(IArray.parray);
SafeArrayUnaccessData(DArray.parray);
}
or
long nNumRecords = UAapi->CopyRetrievedDataToArray((short)0,
&DateArray, &DOWArray, &DydmArray,
&OpenArray, &HighArray, &LowArray, &CloseArray,
&ClosingBidArray, &ClosingAskArray,
&VolumeArray, &OIArray, &TotalVolumeArray, &TotalOIArray, &CashArray);
SafeArrayAccessData(DateArray.parray,(void**)&cDateArray);
SafeArrayAccessData(DydmArray.parray,(void**)&cDydmArray);
SafeArrayAccessData(OpenArray.parray,(void**)&cOpenArray);
SafeArrayAccessData(HighArray.parray,(void**)&cHighArray);
SafeArrayAccessData(LowArray.parray,(void**)&cLowArray);
SafeArrayAccessData(CloseArray.parray,(void**)&cCloseArray);
SafeArrayAccessData(VolumeArray.parray,(void**)&cCVArray);
SafeArrayAccessData(OIArray.parray,(void**)&cCIArray);
for(int j=0;j<nNumRecords;j++)
{
double o=cOpenArray[j];
double h=cHighArray[j];
double l=cLowArray[j];
double c=cCloseArray[j];
fprintf(ptr,”%d,%d, %2.8f00, %2.8f00, %2.8f00, %2.8f00,%8d,%8d\n”,
cDateArray[j], cDydmArray[j],
o,
h,
l,
c,
cCVArray[j],
cCIArray[j] );
}
SafeArrayUnaccessData(DateArray.parray);
SafeArrayUnaccessData(DydmArray.parray);
SafeArrayUnaccessData(OpenArray.parray);
SafeArrayUnaccessData(HighArray.parray);
SafeArrayUnaccessData(LowArray.parray);
SafeArrayUnaccessData(CloseArray.parray);
SafeArrayUnaccessData(VolumeArray.parray);
SafeArrayUnaccessData(OIArray.parray);
Question: I am confused about the use of letters “C” and “X” in relation to the file name of a detrended contract. Which is correct, please?
Answer: There are 2 ways to Detrend, the one that refers to is where the detrend begins back before the current contract. This is the 2nd contract option.
The other begins backwards before the current day, this is the 2nd day option. It looks like this one isn’t documented but may be what you are considering.
Question: We installed CSI “Unfair Advantage 2.10.7.115” on a new Windows 7 Enterprise 64 bit machine. The application works, but we are having problems with API2. The open/high/low/close values come out garbled in our extract program. API2 works fine with our existing Windows XP 32 bit with “Unfair Advantage 2.10.7.7”. What is the issue?
Answer: Versions after 2.10.7.7 have changed the variant data type from float to double due to overflows. This is contained in the CopyDataToRetrievedArray routines.
This may be what you’ve encountered and it’s a good chance that it is.
Question: If I want to get a quote for MSFT, do I need to have MSFT in some kind of portfolio or list a-priori or can I just query the API ad-hoc. If the latter, then does that just retrieve the data over the wire and cache it in the database? My symbol list changes from week-to-week and it would be extremely painful for me to provide it a prior.
Answer: The best way to get a quote in an ad hoc way would be to use either
RetrieveSnapShot or
RetrieveAdjAndUnadjStock( long MarketNumber/*[in]*/,
long LoadStartDate/*[in]*/,
long LoadEndDate/*[in]*/,
VARIANT* IntResultArray/*[out]*/,
VARIANT* FltResultArray/*[out]*/,
long* NumRecords/*[out,retval]*/) // [85]
These return data in 2 variant arrays, an integer for dates, volumes, and a double float for pricing.
Question: I’d like to start pulling future data from CSI through the API and I can’t find any documentation outlining how to access CSI through Python or Java.
Could you provide me some guidance?
Answer: You’re right, I don’t have examples using those languages. I did find these for python, which is where you’ll need to start if you don’t have COM support:
http://www.boddie.org.uk/python/COM.html
or
http://sourceforge.net/projects/pywinauto/
From there, you may look at the way perl connects, and imitate. There is an Exmple.pl in the uapi root.
I’m not up on Python, but with Perl, I found that it does not support Variant array passing through COM. Therefore the return of data can only be done through the Clipboard, which is not advisable. I’d recommend you look into that before tackling this in Python.
As for Java, if J++ is still around, that might work, but I’m not sure about Sun Java. This looks promising:
http://edn.embarcadero.com/article/33305
But again, you may wish to check on the Variant Array passing to COM.
I just perform another search, and found that Perl now supports this. Here is what the lingo talks about:
Variants by reference
Some OLE servers expect parameters passed by reference so that they can be changed in the method call. This allows methods to easily return multiple values. There is preliminary support for this in the Win32::OLE::Variant module:
my $x = Variant(VT_I4|VT_BYREF, 0);
my $y = Variant(VT_I4|VT_BYREF, 0);
$Corel->GetSize($x, $y);
print “Size is $x by $y\n”;
After the GetSize method call $x and $y will be set to the respective sizes. They will still be variants. In the print statement the overloading converts them to string representation automatically.
VT_BYREF is now supported for all variant types (including SAFEARRAYs). It can also be used to pass an OLE object by reference:
my $Results = $App->CreateResultsObject;
$Object->Method(Variant(VT_DISPATCH|VT_BYREF, $Results));
so you could then POSSIBY translate something like this to get the data after
the requests:
$DateArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$OpenArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$HighArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$LowArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$CloseArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$DOWArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$DydmArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$ClosingBidArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$ClosingAskArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$VolumeArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$OIArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$TotalVolumeArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$TotalOIArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$CashArray = Win32::OLE::Variant->new(VT_I4|VT_BYREF, 0);
$numRec= $CSI_OLEhandle->CopyRetrievedDataToArray(0,
\$DateArray,
\$DOWArray,
\$DydmArray,
\$OpenArray,
\$HighArray,
\$LowArray,
\$CloseArray,
\$ClosingBidArray,
\$ClosingAskArray,
\$VolumeArray,
\$OIArray,
\$TotalVolumeArray,
\$TotalOIArray,
\$CashArray);
Question: I am currently developing an application in Excel (2007) to access Unfair Advantage using the API2.
The purpose of the Excel application is to generate a report for a selected list of markets that includes the most recent rolled contract, the current contract and the next available contract. Once if have these contracts identified, I will extract specific contract information such as volume and open interest among other things. I have completed the VBA code to generate a list of basic market information using the Market Profile function, such as market name, exchange, trade units, contract size, valid months, etc. This was rather straight forward. I also use the Build Contract List function to get the number of contracts for a particular market. Now my primary question relates to using the Retrieve Back Adjusted Contract function. I define the various roll parameters such as the market number, roll logic type, roll when, valid months, roll at least N days before expiration, etc. I need to extract is the most recent contract roll date (“Last Roll Date”). Since this function returns the number of data days found, I am guessing that the most recent roll date is somewhere within this data, but I’m not sure where I might find it. How do I determine the most recent roll date?
Answer: You’ll need to iterate through the data to detect the last roll date. Within the data is the contract month field, just go to the end, and compare backwards until you find the change in delivery.
Question: Is the CSI API available for a C++ compilation?
Answer: Yes, the API is available for C++.
Question: I’m trying to access stock prices via the OLE API. This code tries to access IBM prices for all dates:
long count = ua->RetrieveSnapShot(5159, 1, 0, 0, 0, -1, -1, &longArray, &floatArray);
The results include many rows for the sames date (with a different returned market number). If I only use the returned 5159 dates, I still get different results from the Unfair Advantage application.
For date 19620102: UA close=2.64101139, API close=-24.936167.
Why are some API prices negative? Can you provide an API code example that returns the same prices for IBM that are displayed in Unfair Advantage?
Answer: For Stocks you may wish to try using
count = UAapi->RetrieveAdjAndUnadjStock(5159, -1, -1, &IArray, &DArray);
or you can stick with
count= UAapi->RetrieveSnapShot(5159 ,1 , 0 , 0 , 0 , -1 , -1 , &IArray , &DArray);
There are also some properties to consider:
UAapi->SetApplyStockSplitAdjustments(1);
UAapi->SetRoundToTick(0);
UAapi->SetPropStockAdjustments(0);
UAapi->SetApplyStockDividendAdjustments(0);
Here is sample array access code:
VARIANT IArray, DArray;
VariantInit(&IArray);
VariantInit(&DArray);
count = UAapi->RetrieveAdjAndUnadjStock(5159, -1, -1, &IArray, &DArray);
if( count )
{
char strLineOut[128];
long NumIntColumns, NumFltColumns;
{ SafeArrayGetUBound(IArray.parray,1,&NumIntColumns); NumIntColumns++;
SafeArrayGetUBound(DArray.parray,1,&NumFltColumns); NumFltColumns++;
}
long *iArray; SafeArrayAccessData(IArray.parray,(void**)&iArray);
float *dArray; SafeArrayAccessData(DArray.parray,(void**)&dArray);
for(int c=0; c < count; c++ )
{
#define sGetDate(c) iArray[NumIntColumns*(c)+0]
#define sGetDyDm(c) iArray[NumIntColumns*(c)+1]
#define sGetStrike(c) iArray[NumIntColumns*(c)+2]
#define sGetVol(c) iArray[NumIntColumns*(c)+3]
#define sGetTVol(c) iArray[NumIntColumns*(c)+5]
#define sGetOi(c) iArray[NumIntColumns*(c)+4]
#define sGetTOi(c) iArray[NumIntColumns*(c)+6]
#define sGetfStrike(c)dArray[NumFltColumns*(c)+0]
#define sGetOpen(c) dArray[NumFltColumns*(c)+1]
#define sGetHigh(c) dArray[NumFltColumns*(c)+2]
#define sGetLow(c) dArray[NumFltColumns*(c)+3]
#define sGetClose(c) dArray[NumFltColumns*(c)+4]
#define sGetBid(c) dArray[NumFltColumns*(c)+5]
#define sGetAsk(c) dArray[NumFltColumns*(c)+6]
// print to file the commod code
if( isOption )
{ sprintf(strLineOut, “%d,%d,%d,,%d,%g,%g,%g,%g,%g,%g,%d,%d,%d,%d,%d\n”, nMarketNo, sGetDyDm(c),
sGetStrike(c), sGetDate(c),
sGetOpen(c), sGetClose(c),
sGetHigh(c), sGetLow(c),
sGetBid(c), sGetAsk(c),
sGetVol(c), sGetTVol(c),
sGetOi(c), sGetTOi(c),
nOptionConversionFactor);
fputs(strLineOut, pContractsOut);
} else
{ sprintf(strLineOut, “%d,%d,,%d,%g,%g,%g,%g,%g,%g,%d,%d,%d,%d,%d\n”, nMarketNo, sGetDyDm(c),
sGetDate(c),
sGetOpen(c), sGetClose(c),
sGetHigh(c), sGetLow(c),
sGetBid(c), sGetAsk(c),
sGetVol(c), sGetTVol(c),
sGetOi(c), sGetTOi(c),
nConversionFactor);
fputs(strLineOut, pUnderlyingContractsOut);
}
}
SafeArrayUnaccessData(IArray.parray);
SafeArrayUnaccessData(DArray.parray);
}
}
ua->Release();
flcose(ptr);
Question: I am about to upgrade some software which interfaces with the Metastock format. How can I do some ‘C’ routines which can read Microsoft binary format (MBF)?
Answer: It is contained in the m2i_i2m.dll that is located in the ua\uapi directory. This will translate the Microsoft binary into ieee float format and vice versus.
Here are the actual routines that can be used.
extern “C”
{
// XXXX XXXX SMMM MMMM MMMM MMMM MMMM MMMM (MSBIN)
// SXXX XXXX XMMM MMMM MMMM MMMM MMMM MMMM (IEEE)
int msbintoieee( float *src4, float *dst4 )
{
unsigned char szIeee[4], szExp;
memcpy( szIeee, src4, 4 );
szExp=szIeee[3];
szExp-=0x02;
if( szExp == 0xFF )
return 1;
if( szIeee[2] & 0x80 )
szIeee[3]=0x80; // S <- S else szIeee[3]=0x00; szIeee[3]|=(szExp>>1); // -> XXXX XXXX
if( szExp & 0x01 )
szIeee[2]|=0x80;
else
szIeee[2]&=0xEF;
memcpy( dst4, &szIeee, 4 );
return 0;
}
// SXXX XXXX XMMM MMMM MMMM MMMM MMMM MMMM (IEEE)
// XXXX XXXX SMMM MMMM MMMM MMMM MMMM MMMM (MSBIN)
int ieeetomsbin( float *src4, float *dst4 )
{
unsigned char szMsbin[4], szExp;
memcpy( szMsbin, src4, 4 );
szExp=szMsbin[3]<<1;>>7; // XXXX XXXX <- szExp+=0x02; if( szExp == 0xFF ) return 1; if( szMsbin[3] & 0x80 ) // S -> S
szMsbin[2]|=0x80;
else
szMsbin[2]&=0x7F;
szMsbin[3]=szExp;
memcpy( dst4, &szMsbin, 4 );
return 0;
}
}
Question: I am running Linux and would like the ability to pull data from CSI via the C API. Is that possible?
Answer: We haven’t had a chance to embark on a Linux compatible UA. You mentioned
that you were able to makes some calls through wine, but had problems with others.
Maybe we can try to facilitate these, although we’d have to reference wine to see if
it is capable of handling variant safearrays in the proper manner.
It was CopyRetrievedDataToArray, and CopyRetrievedDataToArray2 that failed.
Here is the proper way to use it:
VARIANT IArray, DArray;
VariantInit(&IArray);
VariantInit(&DArray);
int count= UAapi->CopyRetrievedDataToArray2(0, &IArray , &DArray);
if( count )
{
char strLineOut[128];
long NumIntColumns, NumFltColumns;
{ SafeArrayGetUBound(IArray.parray,1,&NumIntColumns); NumIntColumns++;
SafeArrayGetUBound(DArray.parray,1,&NumFltColumns); NumFltColumns++;
}
long *iArray; SafeArrayAccessData(IArray.parray,(void**)&iArray);
double *dArray; SafeArrayAccessData(DArray.parray,(void**)&dArray);
for(int c=0; c < count; c++ )
{
#define sGetDate(c) iArray[NumIntColumns*(c)+0]
#define sGetDyDm(c) iArray[NumIntColumns*(c)+1]
#define sGetStrike(c) iArray[NumIntColumns*(c)+2]
#define sGetVol(c) iArray[NumIntColumns*(c)+3]
#define sGetTVol(c) iArray[NumIntColumns*(c)+5]
#define sGetOi(c) iArray[NumIntColumns*(c)+4]
#define sGetTOi(c) iArray[NumIntColumns*(c)+6]
#define sGetfStrike(c)dArray[NumFltColumns*(c)+0]
#define sGetOpen(c) dArray[NumFltColumns*(c)+1]
#define sGetHigh(c) dArray[NumFltColumns*(c)+2]
#define sGetLow(c) dArray[NumFltColumns*(c)+3]
#define sGetClose(c) dArray[NumFltColumns*(c)+4]
#define sGetBid(c) dArray[NumFltColumns*(c)+5]
#define sGetAsk(c) dArray[NumFltColumns*(c)+6]
fprintf(ptr,”%d, %g, %g, %g, %g\n”, sGetDate(c), sGetOpen(c),sGetLow(c),sGetHigh(c),sGetClose(c) );
}
SafeArrayUnaccessData(IArray.parray);
SafeArrayUnaccessData(DArray.parray);
}
Question: I would like to retrieve data for the Hong Kong Dollar (QE7) like in the code below. I am not sure what call to make to actually retrieve the data. The functions RetrieveStock and RetrieveContract do not seem appropriate here; the former is for Stocks and the latter is for Futures. Is there an appropriate retrieve function for cash, spot, or FX data?
Dim UA As Object
Dim NumBars As Long
Dim ErrNo As Integer
Set UA = CreateObject(“UA.Api2”)
UA.MarketSymbol = “QE7”
UA.MarketNumber = UA.FindMarketNumber()
‘ This will return MarketNumber = 2257…
MsgBox “MarketNumber = ” & UA.MarketNumber
ErrNo = UA.GetMarketProfile()
‘ This will return ErrNo = 1…
MsgBox “ErrNo = ” & ErrNo
‘ This will return QE7 : CASH: Hong Kong Dollars per 1 U.S. Dollar-#390(44)…
MsgBox UA.MarketSymbol + ” : ” + UA.Exchange + ” : ” + UA.MarketName
Answer: You would use RetrieveContract or RetrieveSnapshot for these but specify
“54” as the delivery month. Kind of non-standard but that’s our cash code
here at CSI.
Question: We are using the CSI API2 to extract daily options on futures prices. Our ‘C’ program uses your OLE interface and is very similar to your example ‘C’ code for retrieving option prices. Unfortunately, it takes 4 – 10 seconds to extract prices for a single option both from the API and using the UA application.
A single futures market, CL, currently has 3169 active options. It takes about 6 hours to retrieve prices for these options. We have 50 additional futures markets that we need option prices, so the total time required exceeds 1 day. This makes it impossible to have option prices for all markets available on a daily basis.
This seems very high for a process that I assumed would be IO bound.
We have a critical need to get daily options prices in a reasonable amount of time. Can you give us a faster way to extract daily prices?
Answer: We recommend using RetrieveSnapshot when accessing option data, as you have seen, the other method is not practical.
Question: This is the typical way to get data from a symbol:
7. symbol = “AMRI”
8. UA.MarketSymbol = symbol
10. MarketNumber = UA.FindMarketNumber()
11. UA.MarketNumber = MarketNumber
12. ErrNo = UA.GetMarketProfile()
etc, etc.
However, AMRI is a symbol name that has been reused and thus exists twice in the DB. CSI API will return a handle to the FIRST one. Now, what is the easiest way to get ALL the handles one by one for symbol “AMRI“?
Answer: In this unfortunate case, you may need to depend in our csi numbers, which are available in the factsheet. You can look up the ones you’re interested in, or iterate through them.
Question: I am needing to make a decision about using the API or pulling the data for stocks, futures, spot etc. from the CSI folders as ASCII. We need to pull the data into a database for processing. Some continuous futures contracts will be required. The guys at CSI did not know if continuous futures contracts would have to be built on each call from the API.
I wanted to ask you if:
1) Is it better to pull data from the ASCII files and use UA to generate them or use the API, and, if API is the better method,
2) you are available for building this for me, or if it is better to have my programmer start from scratch with the API.
I figured much of this has already been written, but the guys at CSI said the only thing they have available is the API doc.
Answer: Continuous contracts are rebuilt on every API call. A mechanism can be made to get around this, by first updating the database, and then parsing the update log to see if either a) correction has been posted to the market(s) you’re tracking or b) a roll has occurred. Then you would call to rebuild, otherwise you can call RetrieveSnaphot to obtain the last day of data.
Some customers are doing this from their software, and their are cases where it’s sensible, but I think in your case it may be easier to just update a portfolio and then have your software read those files, you’ll still need to read the whole file into your database, as continuous
contract data does change back in time, at least back-adjusted ones do.
We do have several example programs that use the API located under your ua\API folder. The API is fairly straight forward, your programmer can always request my assistance if you choose
to go that way.
