C4cURL
From C4 Engine Wiki
Overview
Version 0.1.4
2012.05.21
C4cURL is a free, lightweight integration of the C++ cURL library with the C4 Engine for Windows and Linux. It may also work on Mac, although I have not tested this yet myself. While not supporting everything cURL has to offer, the functionality available is versatile enough to satisfy many common requirements. It is currently developed and maintained by Adam Golden and is not supported or endorsed by Terathon Software. The latest version can be found in the Community Contributed Tools section of the Terathon Forums. Full source code is provided there. Enjoy!
Features
• Easy access to web content by URL, supporting GET and POST submissions.
• Construct queries with AddPostVariable("key", "value"), AddGetVariable("key", "value").
• Multithreading with the C4 Job Manager simply by providing a callback.
• Optional parsing of url-encoded responses to decoded key/value pairs.
• Convenient access to decoded key/value pairs with GetKVP() and GetValue("key").
• Ability to set request headers with AddRequestHeader("value").
Integration of cURL (Required)
Windows
1. Download an appropriate distribution of cURL for your platform and development environment. This Wiki is based on my experience with 32-bit Windows 7 and Visual C++ 2008 Express Edition - although many packages are available. In my case, I used the download wizard at http://curl.haxx.se/dlwiz/, selected "libcurl development", then "Windows / Win32", followed by "MSVC".
2. Create a folder for your cURL files, and extract the zip there. To make further explanation easier, I'll assume C:\dev\libs\cURL\
3. Copy the .DLL files from C:\dev\libs\cURL\ into your C:\Windows\System32\ folder, or add the folder to your system path - whichever is best for you. You will need to include these DLL files with any distribution of your game for the same platform.
4. Go to Tools->Options->Projects and Solutions->VC++ Directories. On the top right of that window, select "Include files", and add your C:\dev\libs\cURL\include\ folder, and under "Library files" add your C:\dev\libs\cURL\lib\Release\ folder.
5. To add the cURL library to your game, go to Project->Properties->Configuration Properties->Linker->Input, and in the Additional Dependancies field, add curllib.lib.
Linux
Being new to Ubuntu I can't help a lot with installing a library other than noting what I [believe] I did to get it myself.
sudo apt-get install libcurl4-gnutls-dev
At first it didn't let me, so after some Googling I did this and tried again, at which point it worked.
sudo apt-get install aptitude
sudo aptitude remove libcurl
I'll polish this section soon with more info (where to link to the cURL package in NetBeans, etc.).
Usage with C4 Leak Detection
In order to use C4 Leak Detection with C4cURL:
1. In C4PrefixWindows.h, add this before the line #include <windows.h>:
#include <iostream> #include <string.h> #include <curl/curl.h> // #include <curl/types.h> // older versions of cURL require this include #include <curl/easy.h>
The integration above does not support FASTBUILD. You will need to set FASTBUILD to 0 near the top of the same file or make additional changes yourself (I haven't documented this yet).
2. In C4PrefixLinux.h, add the same thing before the line #include <sys/stat.h>:
3. You must also enable exception handling, or you'll get lots of warnings (C4530) when compiling. You can read more about this issue at http://www.terathon.com/forums/viewtopic.php?f=14&t=5581&p=49628. In Project Properties (for both Debug and Optimized), go to Configuration Properties, C/C++, Command Line, add /EHsc to the Additional options box. Be sure to add this option for the Engine and your projects. Once you've done this, rebuild the solution.
Integration of C4cURL
To use C4cURL in your game, add C4cURL.h and C4cURL.cpp to your project. You may want to make copies to store with your game code then add those instead, should you wish to make game-specific updates of your own. Be sure to include C4cURL.h in the header of any class you'll be using it from:
#include "C4cURL.h"
Samples
Sample 1 - Text from URL, No Callback
Display content from a remote text file in the Console. As no callback function is provided, your code will wait for a response to the Request before continuing.
C4cURL *http = new C4cURL();
if (http->Available())
{
http->Request("http://c4.keyup.net/curl/example.txt");
if ( http->Successful() )
{
Engine::Report( http->GetData() );
} else {
Engine::Report( "Failed to retrieve content." );
}
} else {
Engine::Report( "cURL could not be initialized." );
}
Sample 2 - Text from URL, With Callback
Your callback function should be declared within applicable class definitions like this:
static void httpCompleteProc(C4cURL *http);
Sample 2 callback function:
void MyClass::httpCompleteProc(C4cURL *http)
{
if ( http->Successful() )
{
Engine::Report( http->GetData() );
} else {
Engine::Report( "Failed to retrieve content." );
}
}
Sample 2 usage:
C4cURL *http = new C4cURL();
if (http->Available())
{
http->Request("http://c4.keyup.net/curl/example.txt", false, &httpCompleteProc);
} else {
Engine::Report( "cURL could not be initialized." );
}
Sample 3 - Parsing Query Strings
The PHP used to return the values in this sample is provided later in this article (Associative Arrays).
Sample 3 callback:
void MyClass::httpCompleteProc(C4cURL *http)
{
if ( http->Successful() )
{
http->Report();
} else {
Engine::Report("Failed to retrieve content.");
}
}
Sample 3 usage:
C4cURL *http = new C4cURL();
if (http->Available())
{
http->Request("http://c4.keyup.net/curl/associative.php", true, &httpCompleteProc);
} else {
Engine::Report( "cURL could not be initialized." );
}
Sample 4 - GET and POST submission
You can add any number of GET or POST variables to a request as illustrated below. To more easily view the results of your query, I've set up a URL which will return your variables to the application, with a GET_ and POST_ prefix on your submitted key names, along with a hello=world. The source for this PHP file is available later in the article (Query Echo).
Sample 4 callback:
void MyClass::httpCompleteProc(C4cURL *http)
{
if ( http->Successful() )
{
http->Report();
} else {
Engine::Report("Failed to retrieve content.");
}
}
Sample 4 usage:
C4cURL *http = new C4cURL();
if (http->Available())
{
http->AddPostVariable("mix", "The usual suspects are &, =, +, ', \", \\, and..\nnew lines!\nlike these ones!");
http->AddGetVariable("name", "tester");
http->AddGetVariable("age", "31");
http->Request("http://c4.keyup.net/curl/echo.php", true, &httpCompleteProc);
} else {
Engine::Report( "cURL could not be initialized." );
}
Helpful PHP (used by the samples)
Associative Arrays
Associative arrays are an easy way to set up key/value pairs for encoding. The query string returned in Sample 3 was generated from the associative.php file, shown below. Feel free to open http://c4.keyup.net/curl/associative.php in a web browser to view the encoded values. The PHP urlencode function converts spaces to "+" characters, which must be replaced with "%20" for use with cURL.
<?php
$data = array("message" => "hello world!",
"mix" => "The usual suspects are &, =, +, ', \", \\, and..\nnew lines!\nlike these ones!");
foreach ( $data as $k => $v) echo '&' . $k . '=' . str_replace("+", "%20", urlencode($v));
?>
Query Echo
This is the echo.php file used in Sample 4 to return submitted pairs with a POST_ and GET_ prefix. If you want the pairs returned unmodified, simply remove the line beginning with echo, replace '&POST_' with '&', and '&GET_' with '&'.
<?php echo 'hello=world'; foreach ( $_POST as $k => $v ) echo '&POST_' . $k . '=' . str_replace( "+", "%20", urlencode( stripslashes($v) ) ); foreach ( $_GET as $k => $v ) echo '&GET_' . $k . '=' . str_replace( "+", "%20", urlencode( stripslashes($v) ) ); ?>
Public Functions
AddGetVariable
void AddGetVariable(const char *key, const char *value);
Appends GET variables to the next Request. May be used in combination with AddPostVariable.
http->AddGetVariable("mykey", "and a value");
AddPostVariable
void AddPostVariable(const char *key, const char *value);
Appends POST variables to the next Request. May be used in combination with AddGetVariable.
http->AddPostVariable("username", "the username");
http->AddPostVariable("password", "and the password");
AddRequestHeader
void AddRequestHeader(char *value);
Adds an HTTP Request Header to subsequent requests.
http->AddRequestHeader("User-Agent: C4cURL 0.1.3");
Available
bool Available(void) const;
This should always return true, unless there was an error initializing cURL.
C4cURL *http = new C4cURL();
if (http->Available()
{
// cURL was successfully initialized.
}
Busy
bool Busy(void) const;
Returns true if a request is currently being processed. Useful when multithreading.
if( http->Busy() )
{
// A request is already being processed.
}
GetContentType
String<> GetContentType(void) const;
Returns the content type of the previous response, i.e. "text/html", "application/xml", etc.
Engine::Report( http->GetContentType() );
GetData
String<> GetData(void) const;
Returns the data received during the request. Useful if implementing custom parsing solutions (i.e. for XML), or where decoding is not desired (such as a text file). You should set the autoKVP parameter of the Request to false if you won't be receiving an encoded response (and therefore won't be parsing key/value pairs).
Engine::Report( http->GetData() );
GetKVP
List<C4cURLkvp> * GetKVP(void);
Returns a List of all parsed key/value pairs. Requires autoKVP set to true when calling Request, or that ParseKVP was called after the Request completed. This is the most efficient way to process multiple keys.
String<> str = "KVP:";
C4cURLkvp *p = http->GetKVP()->First();
while (p)
{
str += "\n\nKey: ";
str += p->GetKey();
str += "\nValue: ";
str += p->GetValue();
p = p->ListElement<C4cURLkvp>::Next();
}
Engine::Report(str);
GetPing
unsigned long GetPing(void) const;
Returns the duration of the Request in milliseconds.
String<> str = "The request took "; str += Text::IntegerToString( http->GetPing() ); str += "ms to complete."; Engine::Report( str );
GetResultCode
CURLcode GetResultCode(void) const;
Returns the cURL result code of the previous request.
if (http->GetResultCode() == CURLE_OK)
{
// the request completed successfully.
}
Comparing with CURLE_OK is the equivalent of checking Successful:
if ( http->Successful() )
{
// the request completed successfully.
}
GetValue
String<> GetValue(const char *key) const;
Returns the value of the specified key. Requires autoKVP set to true when calling Request, or that ParseKVP was called after the Request completed. If you'll be processing a significant number of keys, the best performance will come from using GetKVP.
Engine::Report( http->GetValue("mykey") );
ParseKVP
void ParseKVP(void);
void ParseKVP(const char *text);
Calling this function (either by setting autoKVP true with the Request, or after the request completes), will parse the content received to key/value pairs, which are then accessible from GetKVP and GetValue. Query strings which are not properly encoded should not be parsed. Calling this function without a parameter will do the same as if you had called ParseKVP( GetData() ), which will parse the content received by the previous Request. You can also call this function with your own encoded query strings to make use of C4cURL parsing functionality directly. If you're using a callback with your Request and plan to call this, consider using autoKVP instead of calling this function, as parsing will then be done on the seperate thread before completion.
PurgeGetVariables
void PurgeGetVariables(void);
Removes all GET variables added with AddGetVariable.
PurgePostVariables
void PurgePostVariables(void);
Removes all POST variables added with AddPostVariable.
PurgeRequestHeaders
void PurgeRequestHeaders(void);
Removes all HTTP Request Headers added with AddRequestHeader.
RemoveGetVariable
void RemoveGetVariable(const char *key);
Removes a GET variable created with AddGetVariable.
RemovePostVariable
void RemovePostVariable(const char *key);
Removes a POST variable created with AddPostVariable.
ReplaceGetVariable
void ReplaceGetVariable(const char *key, const char *value);
Replaces the value of the specified key created with AddGetVariable.
ReplacePostVariable
void ReplacePostVariable(const char *key, const char *value);
Replaces the value of the specified key created with AddPostVariable.
Request
void Request(const char *url, bool autoKVP = false, void (*completedProc)(C4cURL *c) = nullptr);
Initiates a request to the url specified in the first parameter. Only the first parameter (url) is required. The second parameter (autoKVP) specifies whether to parse the response to key/value pairs automatically or not. If you're dealing with a very large query, it's best to set this to true and use a callback - this way, all the processing is done on a seperate thread. If you set autoKVP to false, you can still parse the response with ParseKVP(), or optionally, ParseKVP(yourText) to populate the KVP List yourself with a user-defined query string.
By providing a callback in the third parameter (completedProc), C4cURL will submit the job to the C4 Job Manager and return immediately, executing your callback on completion. After calling Request, you can use the Busy() function to check whether it's complete or not from elsewhere in your game. For example, if you periodically query the web server for the latest high scores, you could make sure it's not already checking before inadvertantly making another call concurrently for the same reason.
Malformed query string responses may have unexpected results when parsing key/value pairs. Please see the Associative Arrays PHP example above if you're not sure how to form these. Don't set the autoKVP value to true or call ParseKVP unless you're sure the query string returned is properly encoded.
Report
void Report(void) const;
Calls Engine::Report with information about the previous request, as shown in the screenshots for Sample 3 and 4.
Successful
bool Successful(void) const;
Returns true if the previous request's result code was CURLE_OK.
Additional Notes
• Wherever possible, I've used functions and classes which are part of C4 Engine and the cURL library within the C4cURL code. The only other includes are iostream and strings.h, which I believe are necessary. If you have an alternative which is more tightly integrated, or a different approach to something which offers better performance without increasing overhead, please let me know.
• As it stands, this has been tested exclusively in Visual C++ 2008. If you're compiling a Mac version of your game, I'd appreciate some details of your experience with getting this working - hopefully how easy it was haha..
• You can only have one request being processed by cURL at any given time. While you may have multiple copies of C4cURL instantiated, you must ensure that no others are Busy when you call Request.
• Of course, the console window is just for testing! There are dozens of useful applications for this class - signup forms, content management, leaderboards, messages of the day, update notices and so on.
• I plan to integrate more features of the cURL library in my spare time, in the order that's best for my own purposes... but if you have a specific feature request please let me know and I'll consider making it a priority.
Contributions
Anyone who shares improvements which are integrated will have their specific contribution noted here.
Known Issues
If you have a solution to any of these issues, or discover any bugs, please contact me by PM or email!
1. Only ASCII content is supported at this time. Multi-byte encodings will not work.
2. Retrieval of binary files is not supported in this release. The data length given by Report is for ASCII content only (via C4::Text::GetTextLength). I am planning to add file upload and download capabilities in the future.
3. If you receive a warning for macro redefinition of _WIN32_WINNT while compiling, it can be resolved by wrapping the C4PrefixWindows.h definition with a conditional:
#ifndef _WIN32_WINNT
#define _WIN32_WINNT 0x0502
#endif
4. When using a callback (multithreading) within a Debug build, cURL returns CURLE_COULDNT_RESOLVE_HOST instead of CURLE_OK. This issue does not manifest within Optimized builds, and may only be applicable for those using a pre-built release of cURL (unconfirmed). If this happens to you, the solution is to provide a blocking mode fallback for your Debug builds:
#if C4DEBUG http->Request(urlRequest, true); httpCompleteProc(http); #else http->Request(urlRequest, true, httpCompleteProc); #endif
5. C4cURL does not compile in my game project without setting FASTBUILD to 0 in C4PrefixWindows.h. I'm not sure what the cause of this is, or if it's confined to my project specifically (haven't tried with a new project recently), but if anyone finds a workaround please let me know.
Changelog
2012.05.21 - 0.1.4
Update:
• Renamed the "Success" function to "Successful" for Linux build issue.
• Confirmed working on Windows 7 and Linux (Ubuntu) with the latest release of cURL.
• Minor changes since the previous release 2 years ago.
2010.07.19 - 0.1.3
Changed:
• GetValue now returns a String instead of a const char *.
Fixed:
• Ability to compile with C4 Leak Detection. See the "Usage with C4 Leak Detection" section.
• Thanks to the above, it's a great day for C4cURL - now 100% memory leak free!
2010.07.08 - 0.1.2
Added:
• Success function. Returns true if the previous request's result code was CURLE_OK.
• GetContentType function. Returns the content type of the previous response, i.e. "text/html", "application/xml", etc.
Changed:
• GetResult is now GetResultCode.
• GetData now returns a String instead of a const char *.
Fixed:
• Memory leak with content returned by GetData.
• Value returned by Available is now updated after each request.
2010.07.06 - 0.1.1 Initial Release
Added:
• AddRequestHeader
• PurgeRequestHeaders
• RemoveGetVariable
• RemovePostVariable
Fixed:
• Removed std::copy dependancy (deprecated function).
