My favorite 'interruption' ...

Today, I made by mistake an interesting commit ! In fact, I'm working on a simple command line program and that program is design to process data read from the standard input and for my own convenience I added at the beginning of the main a piece of code that some of my teammate didn't know! So let me introduce to you the well known 'interruption 3'.....

#if defined(WIN32) || defined(WIN64)
  __asm int 3;
#endif


int is the asm instruction you use to generate a software interruption. Under windows the interruption 3 execution will trigger a breakpoint and from the outside you will have the usual popup saying:

click "debug" and select debugger you want use :)

It's a trick I often use when I want to also stop at start-up  or at a fixed code point during a dev phase.

Some 'tricks' about ::ftruncate (for windows & linux)

Hi, as today was an interesting development day, where I (with a good advice of one of my teammate) resolved a weird issue, I think it's the good time to write something on that topic because even after some 'Googling' I didn't find anything.

Let's simplify the context.

We just want to open a file, write some text and close it. Later that file will be open again, we will read the text already present and run some text manipulation and rewrite the file.

As the file size and the amount of data we used were small, we use a real simple implementation that we can summarize with the following code.


#include
#include
#include

#include
#include
using namespace std;

int main() {
int fd = -1;
fd = ::open("sample.txt", O_CREAT | O_RDWR, S_IRWXG | S_IRWXO | S_IRWXU);
char* txt = "a small text to fill the file";
::write(fd, txt, strlen(txt));
::close(fd);

fd = ::open("sample.txt", O_CREAT | O_RDWR , S_IRWXG | S_IRWXO | S_IRWXU);
struct stat data;
bool ret = 0 == ::fstat(fd, &data);
if(ret && data.st_size>0)
{
  char* buf = new char[data.st_size+1]; 
  size_t bytesRead = ::read(fd, &buf[0], data.st_size); 
  delete [] buf; 
}
::ftruncate(fd, 0); 
char* txt2 = "another small text to fill the file"; *
::write(fd, txt2, strlen(txt2));
::close(fd); 
 return 0;

Note the call to 'ftruncate' function, we use it to remove the file content. I know that in a real simple code I could have done that in a several others ways (i.e. ::open with O_TRUNC) but consider the fact that our file is shared across multiple process and that the real implementation is a bit more complex.

But let's continue.... The point is that is didn't work as expected. In fact, if the file type was 'ASCII text' after the 1st write, after the second write the file mode changed to 'data' and it wrote the text 'in-binary'. If we don't perform the 'read' or the 'ftruncate' the file type was still the same 'ASCII text'. The solution came from the following: if we 'read' the whole file the reach the end and it may be possible that the file descriptor is in a strange state where it can find the file mode. So we just added a 'seek' before the call to 'ftruncate' and the issues was gone.

i.e. lseek(fd, 0, SEEK_SET);

Easy, but really weird and I didn't find any question and page mentioning that kind of issue ! but there is one :)

That post is related to the question I asked here

ffmpeg: all concatenation methods don't lead to a correct file

As I used recently ffmpeg to concatente some really huge video files coming from a CamCorder, I discovered that even if they looks really similar the "Concat protocol" and "Concat demuxer"describe in "Concatenating media files" didn't lead to the same result and that the "Concat demuxer" approach introduced some weird PTS (presentation time stamp) at files boundaries.


As I spend several hour comparing the different way to do that, I can now recommend to use one of those 2 simple methods if you have to concatenate files with the following properties:

• if they use the same container and codecs
• if they are consecutive in a way their PTS should not have discontinuities

copy /B 1.MTS+2.MTS out.MTS

or

ffmpeg -i "concat:1.MTS|2.MTS" -an -vcodec copy out.ts

 It should be fast and it should not introduce weird PTS between 1 and 2 !

Observed with:

ffmpeg version N-68482-g92a596f Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec 16 2014 02:53:08 with gcc 4.9.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libblu
ray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrw
b --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --
enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-decklink --enable-zlib
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 15.100 / 56. 15.100
  libavformat    56. 15.105 / 56. 15.105
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  3.101 /  5.  3.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100

CMAKE script retro-compatibility

Hi,

I didn't post anything for a while, but in since few days, I found several topic for which I would like to write something (It's like a reminder for me ...)

I worked on product that we build on several different linux platform (Centos/Ubuntu/ etc ...). We use cmake and someone updated a script in a way it couldn't work anymore with older version of cmake (prior to 2.8.3). As I implemented a work-around, I wanted to share it because it was an annoying issue.

In cmake v2.8.3, a new "Useful variable" was introduced CMAKE_CURRENT_LIST_DIR.
If you had to use a cmake script in which that variable is used, cmake will not complain, it will just consider the variable as empty.

The work-around use the other function of cmake to initialize that value and let the rest of your script unchanged.

if("${CMAKE_CURRENT_LIST_DIR}" STREQUAL "")
message(STATUS "work-around to get the current folder for cmake version prior to 2.8.3")
get_filename_component(CMAKE_CURRENT_LIST_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
endif()

It's quit simple, first I test if the value is empty, and if it's true I use get_filename_component cmake command to extract the current folder path from the CMAKE_CURRENT_LIST_FILE cmake variable.

C++ STL copy and lvalue

 

A common mistake when using STL object and algorithm is to assume implementation details. For example when we use the ostringstream class, it may be easy to assume that the object contain a string member and that the str() function must return that data. If we just think like that and don’t look deeper at the str() function declaration, we can write the following code and suppose that the output will be correct.

std::ostringstream os; os << 1234 << std::endl;

std::copy(os.str().begin(), os.str().end(), std::back_inserter(outputContainer));

But it will not be correct and you cannot rely on the compiler to detect the issue because the syntax is correct. We pass 2 iterator on a string, but the issue is that they are not iterator on the ‘same’ string.

The function declaration of str() is:

string str() const;

and it means 2 things:

1. the ostringstream object will not be modified by the call


2. the returned string object is an lvalue (return by copy).

you can also read that post in which that trap is fully detailed….

But to summarize, every call to str() return a different string so the the begin and end iterator we provided in the previous piece of code aren’t related to the same string.

The only correct way to manage that situation is to create a copy of the string like below or to use your own ostringstream version like describe in that post.

 

std::ostringstream os; os << 1234 << std::endl;

std::string acpy = os.str();

std::copy(acpy.begin(), acpy.end(), std::back_inserter(outputContainer));

The conclusion is that when you use a STL object or an algorithm take attention at the declaration because if you don’t you may fall in some traps.

Why sometimes I really hate MACROS

I have sometimes a mixed feeling about MACROS, but today I found 2 reason to hate them a bit more than I like them!

 

I fact I work with a library and I wanted to define a simple C++ mapping table to translate one on that library enum into other value (as int), so I wrote:

static std::map<LibName::Level, int> LibLevelToExternalLevel_Map = boost::assign::map_list_of(LibName::INFO, 40000)(LibName::TRACE, 30000)(LibName::WARNING, 60000)(LibName::CONTROL, 40000/*map on INFO*/)(LibName::ERROR, 70000);

Note that modern C++ provide the initializer list but I’m still in the old age, using C++9… the shame on me !

And to get back on my topic, as I compiled that code, one of my compiler started to complain. From GCC POV that code was ok but from VC98.

(380) : error C2589: 'constant' : illegal token on right side of '::'

As often in that case, you can suspect 2 things:

• compiler bug
• preprocessor bug

I try to find which “LibName::XXXX” label caused the issue and I found that ERROR was the root of the problem….. And after more investigation I found that a windows.h file was include at some point in the .h tree I used for that cpp file.

You may be not aware, but if you include <windows.h> a macro ERROR will be define and it was the root of my issue. A naming collision…. And it’s not the 1st time I had collision in that cpp file. The lib I use also has a class with an API (public function member) call GetMessage, but to build under windows I have to un define a macro for a while….

#ifdef WIN32
#ifdef _UNICODE
#define GetMessage   GetMessageW
#else
#define GetMessage   GetMessageA
#endif
#endif

So to conclude and keep in mind a good rules to prevent that kind of naming conflict or collision, I would just advice follow a simple convention no ‘full’ uppercase in identifier name (function, variable, enum, etc….)  see google coding style

C#: Enum and Attributes …

 

Today I had to update a system with a new kind of device. A device setup is a bit complex and when they start they look into a configuration file and try to transform/cast some “System.Configuration.ConfigurationManager.ConnectionStrings” into their corresponding Enum value in the system.

My Enum looks like:

public enum DeviceType
  {
    [Description("Apple IPod")]
    IPOD = 1,
    [Description("Apple iPad")]
    APPLE_IPAD,
    [Description("Google Glass")]
    GGlass,
    [Description("Surface")]
    MS_SURFACE,
    [Description("MS Surface PRO")]
    MS_SURFACE_PRO,
    [Description("Unknow")]
    UNKNOW = 1000
  }

After the addition of the “Google Glass” device … I quickly discovered that in several place I have to add an additional else in an already too long if-else sequence to handle the conversion of the string in an Enum value.

if(MyDeviceType == Device.IPOD.ToString())
     MyDeviceType = DeviceType.IPOD;
else if(MyDeviceType == Device.APPLE_IPAD.ToString())
     MyDeviceType = DeviceType.APPLE_IPAD;

     etc... 

If think it looks like that just because too many developers doesn’t master the language  they use and first I would say that from my point of view I’m not a master but just a developer aware some useful functionalities of C# and one I like is the template and how you can explore the types/values and check if they have attribute.

So to simply, improve the code and add a bit of flexibility in the configuration string we use I decided to add an “Helpers” method in that system to definitely solve that issue.

public static class Helpers
  {
    public static T GetValueFromDescription<T>(string description)
    {
      var type = typeof(T);
      if (!type.IsEnum) throw new InvalidOperationException();
      foreach (var field in type.GetFields())
      {
          if (field.Name == description)
            return (T)field.GetValue(null);
            
        var attribute = Attribute.GetCustomAttribute(field,
            typeof(DescriptionAttribute)) as DescriptionAttribute;
        if (attribute != null)
        {
          if (attribute.Description == description)
            return (T)field.GetValue(null);
        }
      }
      throw new ArgumentException("Not found.", "description");
      
    }
  }

As you can see, it’s really easy to read. If T is an Enum, we iterate over all field in the Enum. If the string match with a filed name (i.e “IPOD”, “APPLE_IPAD”) it’s OK, we return the field value. If it doesn’t match we look for a Description Attribute. Those attribute are really useful to provide a human readable description for Enum value, class data member, etc… So here if the config string match with a config attribute it’s also OK and due to that “IPOD” and “Apple IPod” return the same Enum value (Flexibility !).

And now if I replace all the existing if-else sequence by something like:

type = DeviceType.UNKNOW;
try
{
  type = ExtensionMethods.GetValueFromDescription<DeviceType>(configuration_string);
}
catch (Exception ) //silently ...
{
  type = DeviceType.UNKNOW;
}

It looks better from my point of view and next time we will a new device type all the mapping code is ready, no update to do !

The only thing I really regret is that all those cool device are only here for the example and I don’t have or work with them …. Snif Snif !