DBGCOOL

debugging seems to be cool, doesn't it?

Browsing Posts in Techs

目前国内各大院校使用的BBS系统,大多是1997年前后从台湾传入大陆的Firebird系统。该系统最早在1990年左右,由Mississippi State University的同学们开发的,原名叫Pirate BBS。这是一种原始的BBS系统,每个连接占用一个TTY终端,所以可以同时在线的人数非常有限。后来在1992年,该系统又被The University of Southern Mississippi大学改为Eagles BBS。这两种BBS都是英文系统,几乎不支持中文。一直到1996年左右,才被台湾国立中正大学汉化,并更名为Firebird BBS,同时也加入了daemon守护进程的支持。

自从1997年BBS系统传入大陆开始,各大院校先后搭建BBS服务器。BBS在线人数也不断超过3000,超过10000,不断创造在线人数新高。南京大学更是开发了njuweb系统,使大家可以通过web浏览器登录BBS系统,查看帖子。在2000年后,BBS系统风靡全国,达到顶峰。BBS系统虽然经过各大院校的不断优化和改造,发展出很多分支,但其核心技术和原型毕竟还是90年代的产物,相对较为原始。

2004年,Facebook的上线标识着新一代的社交网络的开始。Facebook同样出自校园,是由哈佛大学的Mark Zuckerberg同学开发。该系统已经不仅是电子公告板,更增加了很多互动的元素。可以说目前各大院校使用的BBS与Facebook这种模式各有利弊,前者侧重信息共享,以文字为主。后者侧重互动,可以一起游戏、关注好友动态。如果哪个学校可以开发出一套结合二者的系统,或者说可以在现有系统上增加一些互动的元素,那一定可以引领BBS接下来的10年。国内BBS发展慢,通过看BBS的代码可以发现,目前BBS的代码质量不高,Bug很多,系统维护大多是本科生或硕士研究生。本科生刚开始接触C/C++,刚开始接触Linux,刚开始接触网络编程,所以无法写出太高质量的代码。而硕士往往时间有限,没有太多精力维护代码。博士?博士维护BBS系统的人就更少了。

10年后各大院校的BBS会有什么样的发展呢?让我们一起拭目以待吧。

csdn真的很红火,很不错,中国有很多程序员。
并且还有很多人过几年就要成为程序员。

有的人说,IT行业是热门行业,有的人说,IT业干的那么苦,工资那么低。
有的人说,我们找不到工作,有人说,我们招不到人。
有人争论.NET和Java哪个好找工作,有人学习SSH和Ajax,或者DIV+CSS,有人说,这些太难,要转行。
有人说,程序员只能干到30岁,有的人13岁就能写程序了。

这些都对,也都不对,一切的一切,就是因为这里90%的程序员根本就不算程序员。

很多人没有理解HTTP和HTML就来开发Web程序。他们觉得编写程序就是在Toolbox里面挑选控件。
有的人不明白SQL就来访问数据库,DataGridView是他们唯一的法宝,查询数据用它,编辑数据用它,产生报表也用它。
不少人整天把SSH挂在嘴边,但是不知道MVC和IoC是什么,编写程序如同做填空题:这些是自动产生的,这里填入我们的代码。
电脑普及了,电脑学校普及了,程序员也普及了。到处都在做广告,中年主妇都知道Java和C++,不过他们的周围的人也一样知识宽泛但是贫乏。

  • 不会电脑的人不算程序员——很多搞Web开发的人没有上过几个优秀网站;搞桌面开发的Word都用不熟练;甚至连自己电脑里面有没有病毒都不清楚的人不算程序员。(强烈赞成,有太多研发人员连自己的电脑是否中毒都不知道。)
  • 除了C++、Java和.NET以外不知道别的工具的不是程序员。
  • 把C#和ASP.NET画等号的不是程序员。
  • 不理解计算机体系结构的不是程序员。
  • 不理解数据结构的不是程序员。
  • 对设计模式一窍不通的不是程序员。(强烈赞成,太多人认为设计模式是很深奥很神秘的东西,却不知这是基础。不懂设计模式、不懂重构,相信他的代码质量也高不到哪去。)
  • 将设计模式视为圣经,并且过度设计的不是程序员。
  • 没有软件工程意识的程序员不是程序员。
  • 在这里提问却错字连天的不是程序员。人类的语言都掌握不了,面对呆板的计算机,你写的代码会错漏百出。(强烈赞成,错字连天的人不适合做程序员,他写的程序同样漏洞百出。程序员应该具有谨慎、认真的态度。)
  • 在这里提出简单问题,别人给出方法,却连搜索和尝试都懒得做的不是程序员。程序员面对的是挑战性的工作,至少是踏实的工作。(强烈赞成,最讨厌自己连搜索都不搜索就直接问你问题的人。)
  • 在这里寻找免费代码,或者干脆提出一个庞大软件需求,等待别人直接奉献成品的不是程序员。不理解自己工作价值的人,不会干好工作。
  • 不会沟通的人不算程序员,程序员的工作就是帮助一般人和计算机沟通。
  • 用过编程语言,知道很多平台,但是写不出高质量代码的人,不算程序员。(强烈赞成,写不出高质量代码的应该算是程序民工。)
  • 花一个月编写一个简单数据库程序的人不算程序员——他们的开发效率和代码质量不如代码生成器。
  • 收入在5000元以下(发达地区10000元)的不算程序员——程序员是智慧的群体,如果用你的智慧都达不到脑力劳动者的中等收入,怎么能算程序员。
  • 最关键的一点:不热爱这一行的不算程序员。(强烈赞成,一名合格的程序员,必须热爱自己的工作,不停的学习、不停的进步。没激情,混日子过的伪程序员不算程序员。)

大部分程序员进入了高技术企业。
剩下的程序员有的自立门户,有的成为SOHO。
一般的企业当然招不到程序员。

编写了半天的软件,功能上是通用软件的子集,人家软件卖1000元一套。
你当然赚不到钱。

20多岁开始工作,得过且过,混日子的程序员,到了30多岁,连同样混日子,并且年纪小10岁的程序员都比不过,当然要淘汰。

我们社会需要更多的程序员。但是不需要这么多不是程序员的程序员。

这里90%的程序员根本就不算程序员。

原文链接:►►这里90%的程序员根本就不算程序员◄◄

  最近,好多人问我如何通过写个小程序,动态替换可执行文件的图标。这个问题看起来虽小,但却涉及到很多问题。网上也只能找到一些零零散散的资料,却没有详细的指导性文档。所以我决定把这个问题写下来,以方便大家查阅。
  EXE文件图标的替换有很多方法,例如用一个EXE文件的图标替换另外一个EXE文件的图标;用一个ICO文件内的图标替换EXE文件的图标。这两种情况替换的方法不太相同,下面会详细讨论。
  EXE文件图标的替换更一般的情形,是PE(Portable Executable)文件图标的替换。只不过Windows操作系统只会显示EXE文件的图标罢了。但DLL、OCX等PE文件也都可以包含图标资源。下面我们从ICO文件格式说起,一步步讲解替换EXE文件图标的方法和原理。

.ico文件中图标的保存格式

  对于一个扩展名是.ico的文件,大部分人会认为一个ICO文件里面只能包含一个图标。但事实上,一个ICO文件里面可以包含很多图标。而且,目前大部分ICO文件里面都包含有不同尺寸、不同色深的好几个图标。我们以MSN安装包里的msnmsn.ico为例,这个图标文件就包含了9个不同尺寸、不同色深的图标,如图所示:
msnms.ico

  这样做的目的,是为了保证不同的操作系统、不同的桌面色深,图标显示均可达到最佳效果。操作系统会选择并显示一个最合适的图标。Windows XP支持32位色的图标,Windows 2000最多只支持256色的图标。所以,如果我们开发的软件若要同时支持Windows XP和2000,那么为了达到视觉上的最佳效果,每一个ICO文件应至少包含两个图标,一个是32位色的,一个是256色的。 ICO文件头部结构定义如下:


typedef struct
{
    WORD           idReserved;   // Reserved (must be 0)
    WORD           idType;       // Resource Type (1 for icons)
    WORD           idCount;      // How many images?
    ICONDIRENTRY   idEntries[1]; // An entry for each image (idCount of 'em)
} ICONDIR, *LPICONDIR;

idCount表示该ICO文件包含图标的数量,所以理论上,一个ICO文件最多可以包含65535个图标。接下来,是该文件所包含的每一个图标的描述。


typedef struct
{
    BYTE        bWidth;          // Width, in pixels, of the image
    BYTE        bHeight;         // Height, in pixels, of the image
    BYTE        bColorCount;     // Number of colors in image (0 if >=8bpp)
    BYTE        bReserved;       // Reserved ( must be 0)
    WORD        wPlanes;         // Color Planes
    WORD        wBitCount;       // Bits per pixel
    DWORD       dwBytesInRes;    // How many bytes in this resource?
    DWORD       dwImageOffset;   // Where in the file is this image?
} ICONDIRENTRY, *LPICONDIRENTRY;

ICONDIRENTRY中记录了每一个图标的尺寸、色深、图标资源占用的字节数。dwImageOffset是一个文件偏移地址,指向图标资源数据起始位置。至于每一个图标资源内部的具体格式,与本文关系不大,这里就不再详细介绍了。

PE文件中的图标保存格式

  PE文件中的图标保存格式与.ico文件中图标的保存格式略有不同。PE文件中,把ICONDIR和图标资源作为两种资源类型分别保存,前者是RT_GROUP_ICON类型,后者是RT_ICON类型。为了与.ico文件中图标的保存格式做以区分,我们把PE文件中的图标保存格式重新定义如下:


// #pragmas are used here to insure that the structure's
// packing in memory matches the packing of the EXE or DLL.
#pragma pack( push )
#pragma pack( 2 )
typedef struct
{
    WORD              idReserved;   // Reserved (must be 0)
    WORD              idType;       // Resource type (1 for icons)
    WORD              idount;       // How many images?
    GRPICONDIRENTRY   idEntries[1]; // The entries for each image
} GRPICONDIR, *LPGRPICONDIR;

typedef struct
{
    BYTE    bWidth;               // Width, in pixels, of the image
    BYTE    bHeight;              // Height, in pixels, of the image
    BYTE    bColorCount;          // Number of colors in image (0 if >=8bpp)
    BYTE    bReserved;            // Reserved
    WORD    wPlanes;              // Color Planes
    WORD    wBitCount;            // Bits per pixel
    DWORD   dwBytesInRes;         // how many bytes in this resource?
    WORD    nID;                  // the ID
} GRPICONDIRENTRY, *LPGRPICONDIRENTRY;
#pragma pack( pop )

  这里有一个区别,就是在.ico文件中,ICONDIRENTRY结构最后一个成员dwImageOffset表示的是图标资源文件偏移地址。而PE文件中,GRPICONDIRENTRY结构最后一个成员nID表示的是图标的索引ID。

Windows API

  Windows操作系统为我们提供了几个API函数,用来更新PE文件中资源的函数有:BeginUpdateResource, UpdateResource, EndUpdateResource。用来枚举PE文件中资源的函数有:EnumResourceTypes,EnumResourceNames,EnumResourceLanguages。具体的使用方法可以参见MSDN。

下面我们通过具体的例子,来验证上面的方案是否可行。

用一个EXE中的图标替换另外一个EXE文件的图标

在这个例子中,我们用Windows XP自带的记事本的图标替换计算器的图标。

notepad记事本图标

calculator 计算器图标

下面代码演示了如何替换32×32 32bits的图标:


HMODULE hModule = ::LoadLibrary("notepad.exe");
HRSRC hResInfo = ::FindResource(hModule, MAKEINTRESOURCE(8), RT_ICON);
HGLOBAL hGlobal = ::LoadResource(hModule, hResInfo);
DWORD dwSize = ::SizeofResource(hModule, hResInfo);
void* pData = ::LockResource(hGlobal);

HANDLE hUpdate = ::BeginUpdateResource("calc.exe", FALSE);
VERIFY(::UpdateResource(hUpdate, RT_ICON, MAKEINTRESOURCE(7),
                        MAKELANGID(LANG_ENGLISH, SUBLANG_DEFAULT),
                        pData, dwSize));
VERIFY(::EndUpdateResource(hUpdate, FALSE));
VERIFY(::FreeLibrary(hModule));

  大家肯定有个疑问,上面代MAKEINTRESOURCE(8)和MAKEINTRESOURCE(7)是怎么来的呢?其实索引8和7分别是notepad.exe和calc.exe中,32×32 32bits图标的索引。我们可以通过加载RT_GROUP_ICON资源,然后遍历GRPICONDIRENTRY中每一个图标的大小、色深,找到这个图标的索引。为了简便,这里直接写死的索引号,省略了这一动态查找的过程。
  还有一个疑问应该就MAKELANGID(LANG_ENGLISH, SUBLANG_DEFAULT)了。PE文件中,每一个资源都至少对应一种语言。因为我的操作系统是英文的,所以记事本和计算器中的图标资源语言也是英文的。对于简体中文Windows XP操作系统所自带的记事本和计算器,这个值应该是MAKELANGID(LANG_CHINESE, SUBLANG_SYS_DEFAULT)。
  那么我们怎么才能知道一个PE文件中,图标资源的语言是什么呢?我们可以通过资源枚举API,枚举所有图标、语言。可以参考上面提到过的那几个API函数,并查阅MSDN获取这些函数的帮助文档。 我们用记事本32×32 32bits图标替换计算器同样尺寸、色深的图标后,效果如下,在Titles显示方式下,图标大小是48×48的,图标没有被改变:

notepad 48x48
在Icons显示方式下,图标大小是32×32的,图标被我们改变了:
notepad 32x32

用一个ICO文件中的图标替换另外一个EXE文件的图标

  用ICO文件中的图标替换EXE文件图标稍微有点麻烦,我们必须借助数据结构ICONDIR和ICONDIRENTRY来完成。我们使用msnms.ico中的32×32 32bits图标替换计算器中同样大小色深的图标:


DWORD dwSize = sizeof(ICONDIRENTRY);

HANDLE hFile = ::CreateFile("msnms.ico", GENERIC_READ, FILE_SHARE_READ,
                            NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
::SetFilePointer(hFile, sizeof(ICONDIR) + dwSize * 6, NULL, FILE_BEGIN);

DWORD dwRead = 0;
ICONDIRENTRY Entry;
VERIFY(::ReadFile(hFile, &Entry, dwSize, &dwRead, NULL));

::SetFilePointer(hFile, Entry.dwImageOffset, NULL, FILE_BEGIN);

void* pData = new char[Entry.dwImageOffset];
VERIFY(::ReadFile(hFile, pData, Entry.dwBytesInRes, &dwRead, NULL));

HANDLE hUpdate = ::BeginUpdateResource("calc.exe", FALSE);
VERIFY(::UpdateResource(hUpdate, RT_ICON, MAKEINTRESOURCE(7),
                        MAKELANGID(LANG_ENGLISH, SUBLANG_DEFAULT),
                        pData, Entry.dwBytesInRes));
VERIFY(::EndUpdateResource(hUpdate, FALSE));

delete[] pData;
pData = NULL;

VERIFY(::CloseHandle(hFile));

  上面代码中sizeof(ICONDIR) + dwSize * 6的意思是定位到第8个标结构体ICONDIRENTRY的位置,这个图标是32×32 32bits的。我们可以通过遍历每一个ICONDIRENTRY来判断,到底哪个图标是这个尺寸的。这里我们为了简便,把这部分代码省略了。   定位到第8个图标结构体ICONDIRENTRY的位置后,Entry.dwImageOffset的值就是第8个图标资源的文件偏移地址,Entry.dwBytesInRes的值是第8个图标图标资源的大小。然后我们将文件指针定位到Entry.dwImageOffset,并读取Entry.dwBytesInRes大小的数据到指针pData指向的内存当中。 最后,是替换文件图标资源的代码,这部分代码跟上一个例子是相同的。

  本文抛砖引玉,介绍了EXE文件图标的替换,但完全可以推广到所有PE文件图标的替换(包括EXE、DLL等),也可推广到所有PE文件资源的替换(包括图标、图片、文字资源、对话框模板、菜单等)。可供相关人员参考。

C++: Under the Hood

Jan Gray
March 1994
Jan Gray is a Software Design Engineer in Microsoft’s Visual C++ Business Unit. He helped design and implement the Microsoft Visual C++ compiler.

Introduction

It is important to understand how your programming language is implemented. Such knowledge dispels the fear and wonder of “What on earth is the compiler doing here?? imparts confidence to use the new features; and provides insight when debugging and learning other language features. It also gives a feel for the relative costs of different coding choices that is necessary to write the most efficient code day to day.

This paper looks “under the hood?of C++, explaining “run-time?C++ implementation details such as class layout techniques and the virtual function call mechanism. Questions to be answered include:

  • How are classes laid out?
  • How are data members accessed?
  • How are member functions called?
  • What is an adjuster thunk?
  • What are the costs:
    • Of single, multiple, and virtual inheritance?
    • Of virtual functions and virtual function calls?
    • Of casts to bases, to virtual bases?
    • Of exception handling?

First, we’ll look at struct layout of C-like structs, single inheritance, multiple inheritance, and virtual inheritance, then consider data member access and member functions, virtual and not. We’ll examine the workings of constructors, destructors, and assignment operator special member functions and dynamic construction and destruction of arrays. Finally, we’ll briefly consider the impact of exception-handling support.

For each language feature topic, we’ll very briefly present motivation and semantics for the language feature (although “Introduction to C++? this is not), and examine how the language feature was implemented in Microsoft?Visual C++? Note the distinction between abstract language semantics and a particular concrete implementation. Other vendors have sometimes made different implementation choices for whatever reasons. In a few cases we contrast the Visual C++ implementation with others.

Class Layout

In this section we’ll consider the storage layouts required for different kinds of inheritance.

C-like Structs

As C++ is based upon C, it is “mostly?upwards-compatible with C. In particular, the working papers specify the same simple struct layout rules that C has: Members are laid out in their declaration order, subject to implementation defined alignment padding. All C/C++ vendors ensure that valid C structs are stored identically by their C and C++ compilers. Here A is a simple C struct with the obvious expected member layout and padding.


struct A {
    char c;
    int i;
};

C-like Structs with C++ Features

Of course, C++ is an object-oriented programming language: It provides inheritance, encapsulation, and polymorphism by extending the mundane C struct into the wondrous C++ class. Besides data members, C++ classes can also encapsulate member functions and many other things. However, except for hidden data members introduced to implement virtual functions and virtual inheritance, the instance size is solely determined by a class’s data members and base classes.

Here B is a C-like struct with some C++ features: There are public/protected/private access control declarations, member functions, static members, and nested type declarations. Only the non-virtual data members occupy space in each instance. Note that the standards committee working papers permit implementations to reorder data members separated by an access declarator, so these three members could have been laid out in any order. (In Visual C++, members are always laid out in declaration order, just as if they were members of a C struct)


struct B {
public:
    int bm1;
protected:
    int bm2;
private:
    int bm3;
    static int bsm;
    void bf();
    static void bsf();
    typedef void* bpv;
    struct N { };
};

Single Inheritance

C++ provides inheritance to factor out and share common aspects of different types. A good example of a classes-with-inheritance data type organization is biology’s classification of living things into kingdoms, phyla, orders, families, genus, species, and so on. This organization makes it possible to specify attributes, such as “mammals bear live young? at the most appropriate level of classification; these attributes are then inherited by other classes, so we can conclude without further specification that whales, squirrels, and people bear live young. Exceptional cases, such as platypi (a mammal, yet lays eggs), will require that we override the inherited attribute or behavior with one more appropriate for the derived class. More on that later.

In C++, inheritance is specified by using the : base?syntax when defining the derived class. Here D is derived from its base class C.


struct C {
    int c1;
    void cf();
};

struct D : C {
    int d1;
    void df();
};

Since a derived class inherits all the properties and behavior of its base class, each instance of the derived class will contain a complete copy of the instance data of the base class. Within D, there is no requirement that C’s instance data precede D’s. But by laying D out this way, we ensure that the address of the C object within D corresponds to the address of the first byte of the D object. As we shall see, this eliminates adding a displacement to a D* when we need to obtain the address of its embedded C. This layout is used by all known C++ implementations. Thus, in a single inheritance class hierarchy, new instance data introduced in each derived class is simply appended to the layout of the base class. Note our layout diagram labels the “address points?of pointers to the C and D objects within a D.

Multiple Inheritance

Single inheritance is quite versatile and powerful, and generally adequate for expressing the (typically limited) degree of inheritance present in most design problems. Sometimes, however, we have two or more sets of behavior that we wish our derived class to acquire. C++ provides multiple inheritance to combine them.

For instance, say we have a model for an organization that has a class Manager (who delegates) and class Worker (who actually does the work). Now how can we model a class MiddleManager, who, like a Worker, accepts work assignments from his/her manager and who, like a Manager, delegates this work to his/her employees? This is awkward to express using single inheritance: For MiddleManager to inherit behavior from both Manager and Worker, both must be base classes. If this is arranged so that MiddleManager inherits from Manager which inherits from Worker, it erroneously ascribes Worker behavior to Managers. (Vice versa, the same problem.) Of course, MiddleManager could inherit from just one (or neither) of Worker or Manager, and instead, duplicate (redeclare) both interfaces, but that defeats polymorphism, fails to reuse the existing interface, and leads to maintenance woes as interfaces evolve over time.

Instead, C++ allows a class to inherit from multiple base classes:


struct Manager ... { ... };
struct Worker ... { ... };
struct MiddleManager : Manager, Worker { ... };

How might this be represented? Continuing with our “classes of the alphabet?example:


struct E {
    int e1;
    void ef();
};

struct F : C, E {
    int f1;
    void ff();
};

Struct F multiply inherits from C and E. As with single inheritance, F contains a copy of the instance data of each of its base classes. Unlike single inheritance, it is not possible to make the address point of each bases? embedded instance correspond to the address of the derived class:


F f;
// (void*)&f == (void*)(C*)&f;
// (void*)&amp;amp;amp;f <  (void*)(E*)&amp;amp;amp;f;

Here, the address point of the embedded E within F is not at the address of the F itself. As we shall see when we consider casts and member functions, this displacement leads to a small overhead that single inheritance does not generally require.

An implementation is free to lay out the various embedded base instances and the new instance data in any order. Visual C++ is typical in laying out the base instances in declaration order, followed by the new data members, also in declaration order. (As we shall see, this is not necessarily the case when some bases have virtual functions and others don’t).

Virtual Inheritance

Returning to the MiddleManager example which motivated multiple inheritance in the first place, we have a problem. What if both Manager and Worker are derived from Employee?


struct Employee { ... };
struct Manager : Employee { ... };
struct Worker : Employee { ... };
struct MiddleManager : Manager, Worker { ... };

Since both Worker and Manager inherit from Employee, they each contain a copy of the Employee instance data. Unless something is done, each MiddleManager will contain two instances of Employee, one from each base. If Employee is a large object, this duplication may represent an unacceptable storage overhead. More seriously, the two copies of the Employee instance might get modified separately or inconsistently. We need a way to declare that Manager and Worker are each willing to share a single embedded instance of their Employee base class, should Manager or Worker ever be inherited with some other class that also wishes to share the Employee base class.

In C++, this “sharing inheritance?is (unfortunately) called virtual inheritance and is indicated by specifying that a base class is virtual.


struct Employee { ... };
struct Manager : virtual Employee { ... };
struct Worker : virtual Employee { ... };
struct MiddleManager : Manager, Worker { ... };

Virtual inheritance is considerably more expensive to implement and use than single and multiple inheritance. Recall that for single (and multiple) inherited bases and derived classes, the embedded base instances and their derived classes either share a common address point (as with single inheritance and the leftmost base inherited via multiple inheritance), or have a simple constant displacement to the embedded base instance (as with multiple inherited non-leftmost bases, such as E). With virtual inheritance, on the other hand, there can (in general) be no fixed displacement from the address point of the derived class to its virtual base. If such a derived class is further derived from, the further deriving class may have to place the one shared copy of the virtual base at some other, different offset in the further derived class. Consider this example:


struct G : virtual C {
    int g1;
    void gf();
};

struct H : virtual C {
    int h1;
    void hf();
};

struct I : G, H {
    int i1;
    void _if();
};

Ignoring the vbptr members for a moment, notice that within a G object, the embedded C immediately follows the G data member, and similarly notice that within an H, the embedded C immediately follows the H data member. Now when we layout I, we can’t preserve both relationships. In the Visual C++ layout above, the displacements from G to C in a G instance and in an I instance are different. Since classes are generally compiled without knowledge of how they will be derived from, each class with a virtual base must have a way to compute the location of the virtual base from the address point of its derived class.

In Visual C++, this is implemented by adding a hidden vbptr (“virtual base table pointer? field to each instance of a class with virtual bases. This field points to a shared, per-class table of displacements from the address point of the vbptr field to the class’s virtual base(s).

Other implementations use embedded pointers from the derived class to its virtual bases, one per base. This other representation has the advantage of a smaller code sequence to address the virtual base, although an optimizing code generator can often common-subexpression-eliminate repeated virtual base access computations. However, it also has the disadvantages of larger instance sizes for classes with multiple virtual bases, of slower access to virtual bases of virtual bases (unless one incurs yet further hidden pointers), and of a less regular pointer to member dereference.

In Visual C++, G has a hidden vbptr which addresses a virtual base table whose second entry is GdGvbptrC. (This is our notation for “in G, the displacement from G’s vbptr to C? (We omit the prefix to “d?if the quantity is constant in all derived classes.)) For example, on a 32-bit platform, GdGvbptrC would be 8 (bytes). Similarly, the embedded G instance within an I addresses a vbtable customized for G’s within I’s, and so IdGvbptrC would be 20.

As can be seen from the layouts of G, H, and I, Visual C++ lays out classes with virtual bases by:
<ul>
<li>Placing embedded instances of the non-virtually inherited bases first,</li>
<li>Adding a hidden vbptr unless a suitable one was inherited from one of the non-virtual bases,</li>
<li>Placing the new data members declared in the derived class, and, finally,</li>
<li>Placing a single instance of each of the virtually inherited bases at the end of the instance.</li>
</ul>
This representation lets the virtually inherited bases “float?within the derived class (and its further derived classes) while keeping together and at constant relative displacements those parts of the object that are not virtual bases.
<h1>Data Member Access</h1>
Now that we have seen how classes are laid out, let’s consider the cost to access data members of these classes. No inheritance. In absence of inheritance, data member access is the same as in C: a dereference off some displacement from the pointer to the object.
[sourcecode language="cpp"]
C* pc;
pc->c1; // *(pc + dCc1);
[/sourcecode]
Single inheritance. Since the displacement from the derived object to its embedded base instance is a constant 0, that constant 0 can be folded with the constant offset of the member within that base.


D* pd;
pd->c1; // *(pd + dDC + dCc1); // *(pd + dDCc1);
pd->d1; // *(pd + dDd1);

Multiple inheritance. Although the displacement to a given base, or to a base of a base, and so on, might be non-zero, it is still constant, and so any set of such displacements can be folded together into one constant displacement off the object pointer. Thus even with multiple inheritance, access to any member is inexpensive.


F* pf;
pf->c1; // *(pf + dFC + dCc1); // *(pf + dFc1);
pf->e1; // *(pf + dFE + dEe1); // *(pf + dFe1);
pf->f1; // *(pf + dFf1);

Virtual inheritance. Within a class with virtual bases, access to a data member or non-virtually inherited base class is again just a constant displacement off the object pointer. However, access to a data member of a virtual base is comparatively expensive, since it is necessary to fetch the vbptr, fetch a vbtable entry, and then add that displacement to the vbptr address point, just to compute the address of the data member. However, as shown for i.c1 below, if the type of the derived class is statically known, the layout is also known, and it is unnecessary to load a vbtable entry to find the displacement to the virtual base.


I* pi;
pi->c1; // *(pi + dIGvbptr + (*(pi+dIGvbptr))[1] + dCc1);
pi->g1; // *(pi + dIG + dGg1); // *(pi + dIg1);
pi->h1; // *(pi + dIH + dHh1); // *(pi + dIh1);
pi->i1; // *(pi + dIi1);
I i;
i.c1; // *(&amp;amp;amp;i + IdIC + dCc1); // *(&amp;amp;amp;i + IdIc1);

What about access to members of transitive virtual bases, for example, members of virtual bases of virtual bases (and so on)? Some implementations follow one embedded virtual base pointer to the intermediate virtual base, then follow its virtual base pointer to its virtual base, and so on. Visual C++ optimizes such access by using additional vbtable entries which provide displacements from the derived class to any transitive virtual bases.

Casts

Except for classes with virtual bases, it is relatively inexpensive to explicitly cast a pointer into another pointer type. If there is a base-derived relationship between class pointers, the compiler simply adds or subtracts the displacement between the two (often 0).


F* pf;
(C*)pf; // (C*)(pf ? pf + dFC : 0); // (C*)pf;
(E*)pf; // (E*)(pf ? pf + dFE : 0);

In the C* cast, no computations are required, because dFC is 0. In the E* cast, we must add dFE, a non-zero constant, to the pointer. C++ requires that null pointers (0) remain null after a cast. Therefore Visual C++ checks for null before performing the addition. This check occurs only when a pointer is implicitly or explicitly converted to a related pointer type, not when a derived* is implicitly converted to a base*const this pointer when a base member function is invoked on a derived object.

As you might expect, casting over a virtual inheritance path is relatively expensive: about the same cost as accessing a member of a virtual base:


I* pi;
(G*)pi; // (G*)pi;
(H*)pi; // (H*)(pi ? pi + dIH : 0);
(C*)pi; // (C*)(pi ? (pi+dIGvbptr + (*(pi+dIGvbptr))[1]) : 0);

In general, you can avoid a lot of expensive virtual base field accesses by replacing them with one cast to the virtual base and base relative accesses:


/* before: */             ... pi->c1 ... pi->c1 ...
/* faster: */ C* pc = pi; ... pc->c1 ... pc->c1 ...

Member Functions

A C++ member function is just another member in the scope of its class. Each (non-static) member function of a class X receives a special hidden this parameter of type X *const, which is implicitly initialized from the object the member function is applied to. Also, within the body of a member function, member access off the this pointer is implicit.


struct P {
    int p1;
    void pf(); // new
    virtual void pvf(); // new
};

P has a non-virtual member function pf() and a virtual member function pvf(). It is apparent that virtual member functions incur an instance size hit, as they require a virtual function table pointer. More on that later. Notice there is no instance cost to declaring non-virtual member functions. Now consider the definition of P::pf():


void P::pf() { // void P::pf([P *const this])
    ++p1;   // ++(this->p1);
}

Here P::pf() receives a hidden this parameter, which the compiler has to pass each call. Also note that member access can be more expensive than it looks, because member accesses are this relative. On the other hand, compilers commonly enregister this so member access cost is often no worse than accessing a local variable. On the other hand, compilers may not be able to enregister the instance data itself because of the possibility this is aliased with some other data.

Overriding Member Functions

Member functions are inherited just as data members are. Unlike data members, a derived class can override, or replace, the actual function definition to be used when an inherited member function is applied to a derived instance. Whether the override is static (determined at compile time by the static types involved in the member function call) or dynamic (determined at run-time by the dynamic object addressed by the object pointer) depends upon whether the member function is declared virtual.

Class Q inherits P’s data and function members. It declares pf(), overriding P::pf(). It also declares pvf(), a virtual function overriding P::pvf(), and declares a new non-virtual member function qf(), and a new virtual function qvf().


struct Q : P {
    int q1;
    void pf();  // overrides P::pf
    void qf();  // new
    void pvf(); // overrides P::pvf
    virtual void qvf(); // new
};

For non-virtual function calls, the member function to call is statically determined, at compile time, by the type of the pointer expression to the left of the -> operator. In particular, even though ppq points to an instance of Q, ppq->pf() calls P::pf(). (Also notice the pointer expression left of the -> is passed as the hidden this parameter.)


P p; P* pp = &amp;amp;amp;p; Q q; P* ppq = &amp;amp;amp;q; Q* pq = &amp;amp;amp;q;
pp->pf();  // pp->P::pf();  // P::pf(pp);
ppq->pf(); // ppq->P::pf(); // P::pf(ppq);
pq->pf();  // pq->Q::pf();  // Q::pf((P*)pq);
pq->qf();  // pq->Q::qf();  // Q::qf(pq);

For virtual function calls, the member function to call is determined at run-time. Regardless of the declared type of the pointer expression left of the -> operator, the virtual function to call is the one appropriate to the type of the actual instance addressed by the pointer. In particular, although ppq has type P*, it addresses a Q, and so Q::pvf() is called.


pp->pvf();  // pp->P::pvf();  // P::pvf(pp);
ppq->pvf(); // ppq->Q::pvf(); // Q::pvf((Q*)ppq);
pq->pvf();  // pq->Q::pvf();  // Q::pvf((P*)pq);

Hidden vfptr members are introduced to implement this mechanism. A vfptr is added to a class (if it doesn’t already have one) to address that class’s virtual function table (vftable). Each virtual function in a class has a corresponding entry in that class’s vftable. Each entry holds the address of the virtual function override appropriate to that class. Therefore, calling a virtual function requires fetching the instance’s vfptr, and indirectly calling through one of the vftable entries addressed by that pointer. This is in addition to the usual function call overhead of parameter passing, call, and return instructions. In the example below, we fetch q’s vfptr, which addresses Q’s vftable, whose first entry is &Q::pvf. Thus Q::pvf() is called.

Looking back at the layouts of P and Q, we see that the Visual C++ compiler has placed the hidden vfptr member at the start of the P and Q instances. This helps ensure that virtual function dispatch is as fast as possible. In fact, the Visual C++ implementation ensures that the first field in any class with virtual functions is always a vfptr. This can require inserting the new vfptr before base classes in the instance layout, or even require that a right base class that does begin with a vfptr be placed before a left base that does not have one.

Most C++ implementations will share or reuse an inherited base’s vfptr. Here Q did not receive an additional vfptr to address a table for its new virtual function qvf(). Instead, a qvf entry is appended to the end of P’s vftable layout. In this way, single inheritance remains inexpensive. Once an instance has a vfptr it doesn’t need another one. New derived classes can introduce yet more virtual functions, and their vftable entries are simply appended to the end of their one per-class vftable.

Virtual Functions: Multiple Inheritance

It is possible for an instance to contain more than one vfptr if it inherits them from multiple bases, each with virtual functions. Consider R and S:


struct R {
    int r1;
    virtual void pvf(); // new
    virtual void rvf(); // new
};

struct S : P, R {
    int s1;
    void pvf(); // overrides P::pvf and R::pvf
    void rvf(); // overrides R::rvf
    void svf(); // new
};

Here R is just another class with some virtual functions. Since S multiply inherits, from P and R, it contains an embedded instance of each, plus its own instance data contribution, S::s1. Notice the right base R has a different address point than do P and S, as expected with multiple inheritance. S::pvf() overrides both P::pvf() and R::pvf(), and S::rvf() overrides R::rvf(). Here are the required semantics for the pvf override:


S s; S* ps = &amp;amp;amp;s;
((P*)ps)->pvf(); // ((P*)ps)->P::vfptr[0])((S*)(P*)ps)
((R*)ps)->pvf(); // ((R*)ps)->R::vfptr[0])((S*)(R*)ps)
ps->pvf();       // one of the above; calls S::pvf()

Since S::pvf() overrides both P::pvf() and R::pvf(), it must replace their vftable entries in the S vftables. However, notice that it is possible to invoke pvf() both as a P and an R. The problem is that R’s address point does not correspond to P’s and S’s. The expression (R*)ps does not point to the same part of the class as does (P*)ps. Since the function S::pvf() expects to receive an S* as its hidden this parameter, the virtual function call itself must automatically convert the R* at the call site into an S* at the callee. Therefore, S’s copy of R’s vftable’s pvf slot takes the address of an adjuster thunk, which applies the address adjustment necessary to convert an R* pointer into an S* as desired.

In MSC++, for multiple inheritance with virtual functions, adjuster thunks are required only when a derived class virtual function overrides virtual functions of multiple base classes.

Address Points and “Logical This Adjustment”

Consider next S::rvf(), which overrides R::rvf(). Most implementations note that S::rvf() must have a hidden this parameter of type S*. Since R’s rvf vftable slot may be used when this call occurs:


((R*)ps)->rvf(); // (*((R*)ps)->R::vfptr[1])((R*)ps)

Most implementations add another thunk to convert the R* passed to rvf into an S*. Some also add an additional vftable entry to the end of S’s vftable to provide a way to call ps->rvf() without first converting to an R*. MSC++ avoids this by intentionally compiling S::rvf() so as to expect a this pointer which addresses not the S object but rather the R embedded instance within the S. (We call this “giving overrides the same expected address point as in the class that first introduced this virtual function?) This is all done transparently, by applying a “logical this adjustment?to all member fetches, conversions from this, and so on, that occur within the member function. (Just as with multiple inheritance member access, this adjustment is constant-folded into other member displacement address arithmetic.)

Of course, we have to compensate for this adjustment in our debugger.


ps->rvf(); // ((R*)ps)->rvf(); // S::rvf((R*)ps)

Thus MSC++ generally avoids creating a thunk and an additional extra vftable entry when overriding virtual functions of non-leftmost bases.

Adjuster Thunks

As described, an adjuster thunk is sometimes called for, to adjust this (which is found just below the return address on the stack, or in a register) by some constant displacement en route to the called virtual function. Some implementations (especially cfront-based ones) do not employ adjuster thunks. Rather, they add additional displacement fields to each virtual function table entry. Whenever a virtual function is called, the displacement field, which is quite often 0, is added to the object address as it is passed in to become the this pointer:


ps->rvf();
// struct { void (*pfn)(void*); size_t disp; };
// (*ps->vfptr[i].pfn)(ps + ps->vfptr[i].disp);

The disadvantages of this approach include both larger vftables and larger code sequences to call virtual functions.

More modern PC-based implementations use adjust-and-jump techniques:


S::pvf-adjust: // MSC++
this -= SdPR;
goto S::pvf()

Of course, the following code sequence is even better (but no current implementation generates it):


S::pvf-adjust:
this -= SdPR; // fall into S::pvf()
S::pvf() { ... }

Virtual Functions: Virtual Inheritance

Here T virtually inherits P and overrides some of its member functions. In Visual C++, to avoid costly conversions to the virtual base P when fetching a vftable entry, new virtual functions of T receive entries in a new vftable, requiring a new vfptr, introduced at the top of T.


struct T : virtual P {
    int t1;
    void pvf();         // overrides P::pvf
    virtual void tvf(); // new
};

void T::pvf() {
    ++p1; // ((P*)this)->p1++; // vbtable lookup!
    ++t1; // this->t1++;
}

As shown above, even within the definition of a virtual function, access to data members of a virtual base must still use the vbtable to fetch a displacement to the virtual base. This is necessary because the virtual function can be subsequently inherited by a further derived class with different layout with respect to virtual base placement. And here is just such a class:


struct U : T {
int u1;
};

Here U adds another data member, which changes the dP, the displacement to P. Since T::pvf expects to be called with a P* in a T, an adjuster thunk is necessary to adjust this so it arrives at the callee addressing just past T::t1 (the address point of a P* in a T). (Whew! That’s about as complex as things get!)

Special Member Functions

This section examines hidden code compiled into (or around) your special member functions.

Constructors and Destructors

As we have seen, sometimes there are hidden members that need to be initialized during construction and destruction. Worst case, a constructor may perform these activities

If “most-derived,?initialize vbptr field(s) and call constructors of virtual bases.
Call constructors of direct non-virtual base classes.
Call constructors of data members.
Initialize vfptr field(s).
Perform user-specified initialization code in body of constructor definition.
(A “most-derived?instance is an instance that is not an embedded base instance within some other derived class.)

So, if you have a deep inheritance hierarchy, even a single inheritance one, construction of an object may require many successive initializations of a class’s vfptr. (Where appropriate, Visual C++ will optimize away these redundant stores.)

Conversely, a destructor must tear down the object in the exact reverse order to how it was initialized:

Initialize vfptr field(s).
Perform user-specified destruction code in body of destructor definition.
Call destructors of data members (in reverse order).
Call destructors of direct non-virtual bases (in reverse order).
If “most-derived,?call destructors of virtual bases (in reverse order).
In Visual C++, constructors for classes with virtual bases receive a hidden “most-derived flag?to indicate whether or not virtual bases should be initialized. For destructors, we use a “layered destructor model,?so that one (hidden) destructor function is synthesized and called to destroy a class including its virtual bases (a “most-derived?instance) and another to destroy a class excluding its virtual bases. The former calls the latter, then destroys virtual bases (in reverse order).

Virtual Destructors and Operator Delete

Consider structs V and W.


struct V {
    virtual ~V();
};

struct W : V {
    operator delete();
};

Destructors can be virtual. A class with a virtual destructor receives a hidden vfptr member, as usual, which addresses a vftable. The table contains an entry holding the address of the virtual destructor function appropriate for the class. What is special about virtual destructors is they are implicitly invoked when an instance of a class is deleted. The call site (delete site) does not know what the dynamic type being destroyed is, and yet it must invoke the appropriate operator delete for that type. For instance, when pv below addresses a W, after W::~W() is called, its storage must be destroyed using W::operator delete().


V* pv = new V;
delete pv;   // pv->~V::V(); // use ::operator delete()
pv = new W;
delete pv;   // pv->~W::W(); // use W::operator delete()
pv = new W;
::delete pv; // pv->~W::W(); // use ::operator delete()

To implement these semantics, Visual C++ extends its “layered destructor model?to automatically create another hidden destructor helper function, the “deleting destructor,?whose address replaces that of the “real?virtual destructor in the virtual function table. This function calls the destructor appropriate for the class, then optionally invokes the appropriate operator delete for the class.

Arrays

Dynamic (heap allocated) arrays further complicate the responsibilities of a virtual destructor. There are two sources of complexity. First, the dynamic size of a heap allocated array must be stored along with the array itself, so dynamically allocated arrays automatically allocate extra storage to hold the number of array elements. The other complication occurs because a derived class may be larger than a base class, yet it is imperative that an array delete correctly destroy each array element, even in contexts where the array size is not evident:


struct WW : W { int w1; };
pv = new W[m];
delete [] pv; // delete m W's  (sizeof(W)  == sizeof(V))
pv = new WW[n];
delete [] pv; // delete n WW's (sizeof(WW) >  sizeof(V))

Although, strictly speaking, polymorphic array delete is undefined behavior, we had several customer requests to implement it anyway. Therefore, in MSC++, this is implemented by yet another synthesized virtual destructor helper function, the so-called “vector delete destructor,?which (since it is customized for a particular class, such as WW) has no difficulty iterating through the array elements (in reverse order), calling the appropriate destructor for each.

Exception Handling

Briefly, the exception handling proposal in the C++ standards committee working papers provides a facility by which a function can notify its callers of an exceptional condition and select appropriate code to deal with the situation. This provides an alternative mechanism to the conventional method of checking error status return codes at every function call return site.

Since C++ is object-oriented, it should come as no surprise that objects are employed to represent the exception state, and that the appropriate exception handler is selected based upon the static or dynamic type of exception object “thrown.?Also, since C++ always ensures that frame objects that are going out of scope are properly destroyed, implementations must ensure that in transferring control (unwinding the stack frame) from throw site to “catch?site, (automatic) frame objects are properly destroyed.

Consider this example:


struct X { X(); }; // exception object class
struct Z { Z(); ~Z(); }; // class with a destructor
extern void recover(const X&amp;amp;amp;);
void f(int), g(int);

int main() {
    try {
        f(0);
    } catch (const X&amp;amp;amp; rx) {
    recover(rx);
}
return 0;
}

void f(int i) {
    Z z1;
    g(i);
    Z z2;
    g(i-1);
}

void g(int j) {
    if (j < 0)
        throw X();
}

This program will throw an exception. main() establishes an exception handler context for its call to f(0), which in turn constructs z1, calls g(0), constructs z2, and calls g(-1). g() detects the negative argument condition and throws an X object exception to whatever caller can handle it. Since neither g() nor f() established an exception handler context, we consider whether the exception handler established by main() can handle an X object exception. Indeed it can. Before control is transferred to the catch clause in main(), however, objects on the frame between the throw site in g() and the catch site in main() must be destroyed. In this case, z2 and z1 are therefore destroyed.

An exception handling implementation might employ tables at the throw site and the catch site to describe the set of types that might catch the thrown object (in general) and can catch the thrown object at this specific catch site, respectively, and generally, how the thrown object should initialize the catch clause “actual parameter.?Reasonable encoding choices can ensure that these tables do not occupy too much space.

However, let us reconsider function f(). It looks innocuous enough. Certainly it contains neither try, catch, nor throw keywords, so exception handling would not appear to have much of an impact on f(). Wrong! The compiler must ensure that, once z1 is constructed, if any subsequently called function were to raise an exception (“throw? back to f(), and therefore out of f(), that the z1 object is properly destroyed. Similarly, once z2 is constructed, it must ensure that a subsequent throw is sure to destroy z2 and then z1.

To implement these “unwind semantics,?an implementation must, behind the scenes, provide a mechanism to dynamically determine the context (site), in a caller function, of the call that is raising the exception. This can involve additional code in each function prolog and epilog, and, worse, updates of state variables between each set of object initializations. For instance, in the example above, the context in which z1 should be destroyed is clearly distinct from the subsequent context in which z2 and then z1 should be destroyed, and therefore Visual C++ updates (stores) a new value in a state variable after construction of z1 and again after construction of z2.

All these tables, function prologs, epilogs, and state variable updates, can make exception handling functionality a significant space and speed expense. As we have seen, this expense is incurred even in functions that do not employ exception handling constructs.

Fortunately, some compilers provide a compilation switch and other mechanisms to disable exception handling and its overhead from code that does not require it.

Summary

There, now go write your own compiler.

Seriously, we have considered many of the significant C++ run-time implementation issues. We see that some wonderful language features are almost free, and others can incur significant overhead. These implementation mechanisms are applied quietly for you, behind the curtains, so to speak, and it is often hard to tell what a piece of code costs when looking at it in isolation. The frugal coder is well advised to study the generated native code from time to time and question whether use of this or that particularly cool language feature is worth its overhead.

Acknowledgments. The Microsoft C++ Object Model described herein was originally designed by Martin O’Riordan and David Jones; yours truly added details here and there as necessary to complete the implementation.

——————————-

WARRANTY DISCLAIMER

THESE MATERIALS ARE PROVIDED “AS-IS”, FOR INFORMATIONAL PURPOSES ONLY. NEITHER MICROSOFT NOR ITS SUPPLIERS MAKES ANY WARRANTY, EXPRESS OR IMPLIED, WITH RESPECT TO THE CONTENT OF THESE MATERIALS OR THE ACCURACY OF ANY INFORMATION CONTAINED HEREIN, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. BECAUSE SOME STATES/JURISDICTIONS DO NOT ALLOW EXCLUSIONS OF IMPLIED WARRANTIES, THE ABOVE LIMITATION MAY NOT APPLY TO YOU.

NEITHER MICROSOFT NOR ITS SUPPLIERS SHALL HAVE ANY LIABILITY FOR ANY DAMAGES WHATSOEVER, INCLUDING CONSEQUENTIAL INCIDENTAL, DIRECT, INDIRECT, SPECIAL, AND LOSS PROFITS. BECAUSE SOME STATES/JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF CONSEQUENTIAL OR INCIDENTAL DAMAGES, THE ABOVE LIMITATION MAY NOT APPLY TO YOU. IN ANY EVENT, MICROSOFT’S AND ITS SUPPLIERS’ ENTIRE LIABILITY IN ANY MANNER ARISING OUT OF THESE MATERIALS, WHETHER BY TORT, CONTRACT, OR OTHERWISE, SHALL NOT EXCEED THE SUGGESTED RETAIL PRICE OF THESE MATERIALS.

This article was originally posted on blogspot.com on April 4, 2009.

Recently I’ve written a program. With the help of this program people can look inside Pal4’s database file. I give it a name which is called Pal4 Database Browser. You can open a Pal4’s .db file by clicking the file-open menu. Here’s a link to this program: Pal4DbBrowser.

You can download .db files here or extract .db files manually yourself from Pal4\gamedata\database.cpk with the help of WinCpk4.

Here’s a screen shot of Pal4DbBrowser: