| |
|
Tor
J. Larsen, Ph.D.
Associate Professor
Despite of its long-term success, it seems certain that investment in information systems (IS) remains a risky business (Rifkin 1995; Sauer 1999). It may be the case that organizations that invest heavily in IS also ripe benefits beyond what is found in organizations with more modest investment portfolios (Hitt and Brynjolfsson 1996). Keeping to the level of projects in systems development efforts, attempts have been made to develop a coherent managerial framework for major and minor issues that are targeted for change or where change may, oftentimes unplanned, occur (Larsen, 1999).
It may be that a key aspect of
the challenge facing us is that the development of IS is looked upon as
an engineering task when it in fact is best understood as a process
with moving targets (Checkland and Howell 1998). This is the core theme
addressed in the MIS Research Topic seminar, Fall 2004. Student groups
select their own issue under the broad umbrella defined above. The
present web-book contains the result of the groups’ endeavor.
References
Checkland, P.B. and Howell, S., Information, Systems and Information Systems – Making Sense of the Field, Chichester: England, John Wiley & Sons, 1998.
Hitt, L.M. and Brynjolfsson, E., “Productivity, Business Profitability and Consumer Surplus: Three Different Measures of Information Technology Value,” MIS Quarterly, [20:2], June 1996, pp. 121-142.
Larsen, T.J. “Line Managers’ Supervision of the Development and Use of Information Systems (IS): Administration through Planning and Control of IS Effects,” in Khosrowpour, M., (Ed.), Proceedings of the 1999 Information Resources Management Association International Conference on Managing Information Technology Resources in Organizations in the Next Millennium, Hersey, PA, May 16-16, 1999, pp. 853-860.
Rifkin, J. The End of Work: The Decline of the Global Labor Force and the Dawn of the Post-Market Era, New York, G.P. Putnam’s Sons, 1995.
Sauer, C., “Deciding the Future
for IS Failures: Not the Choice You Might Think,” in Currie, W. and
Galliers, B., (Eds.), Rethinking
Management Information Systems: An
Interdisciplinary Perspective, Oxford:
England, Oxford University Press, 1999, pp. 279-309.
The impact of information technology (IT)
investments on firm performance has been the subject of active research
recent years. One central question is whether the tremendous amount of
IT capital invested in the last few decades has had any impact on the
performance of the investing firms. Tam. (1998), argues that empirical
findings reported in the literature are mixed. These discrepancies, in
many cases, can be traced back to the inconsistency of performance
measures. This is clarified by Hitt and Brynjolfsson (1996), which
argued that IT value can be measured into three dimensions:
productivity, business performance, and consumer value. Furthermore,
determining whether investments in information technology (IT) have an
impact on firm performance has been and continues to be a major problem
for information systems researchers and practitioners.
When firms make IT investments, the investments result in some direct
benefits that contribute to future cash flows. In addition, the
investments may also have indirect benefits in the form of new
investment opportunities for the firms. For example, investment in a
new technology project may improve a firm’s ability to use this new
technology in future projects, thus affecting the firm’s future
investment opportunities (Dos Santos, 1991). One way that the
undervaluation of IT investments can be overcome is to determine how IT
investments affect the value of the firm. If the net discounted cash
flows that will result from an investment, the net present value (NPV),
are positive because the resulting direct and indirect benefits are
expected to generate a return which is greater than the required rate
of return, then the value of the firm should rise. This change in value
will then be reflected in the market prices of the firm’s securities.
If the firm’s securities are traded in an efficient market,
this change in value will occur very quickly, allowing it to be
observed and measured (Dos Santos et al, 1993).
The extent to which IT systems, and especially
ERP
systems, increase firm value is a fruitful area of inquiry for
accounting
researchers (Hayes et al. 2001). Yet to our knowledge, no accounting
research
has addressed the salient ERP topics, such as the costs and benefits of
system
implementations, the impact of ERP systems on the accounting and
auditing
profession, and the importance of ERP systems to firm valuation and
market
returns. Prior research conducted elswehere than in Norway, has to an
extent
investigated the last mentioned issue.
Therefore, the objective of this research is to examine how the capital
market responds when a firm announces that it plans to implement an
enterprise resource planning (ERP) system. The choice of one particular
system is the rationale of limitate the research. A conduct of various
IS systems would be too large. This study addresses the question: what
are the relationship between contextual factors (firm size and health)
and vendor effects (large as opposed to small), and the market reaction
to firms announcing implementation of ERP systems. To answer this
question, we analyze the impact of ERP implementation announcements on
the common stock prices of publicly-traded firms. We adopt event-time
methodology, which is a standard methodology in the accounting and
finance literature (Loderer and Mauer, 1992). If the market responds by
revaluing the firm’s shares to reflect the information in the
investment announcement, we can conclude that the announcement of an
ERP implementation affects the market value of the firm. More
precisely, if the market expects an IT investment to have a positive
net effect on the value of the firm, then that value will be reflected
in the market price of the firm’s common stock. Moreover, the
consequence leads us to be able to measure the markets assessment
impact of ERP investment on total firm value by examining stock price
reactions around announcements of ERP implementations or investment.
On the other hand, the market’s reaction to ERP implementation
announcements may depend on other factors. Contextual factors such as
firm size and health, and vendor effects are examples of factors that
may have an impact on the market’s reaction to ERP implementation
announcements (Hayes et al, 2001). Our research will be conducted in
the Norwegian market considering if there are similarities, since most
studies have been conducted in the United States. Little work has been
done elsewhere in order to validate these results across national
boundaries (Tam, 1998). We will try to fill this gap by analyzing the
Norwegian market with regard to market reaction to ERP implementation
announcements.
Markets perceive ERP implementation announcements positively and therefore contextual factors such as firm size and health will interact and drive the market’s reaction in different paths as described in theory and hypotheses. Vvendor effects with regard to size and recognition of the ERP vendor will affect the markets reaction. Our aim is to find the relationship among aforementioned factors and the markets reaction to ERP implementation announcements. Thus or research question is
RQ: What are the relationships between contextual factors
and vendor effects, and market reaction to the announcements of
an
ERP implementation?
If there is a relationship between contextual factors and vendor
effects, and the markets reaction to ERP implementation announcement,
then the implication of this research is that investing in IT has an
affect on firm performance, with regard to market valuation of the firm.
The chapter will now continue with a presentation
of the
literatur review within ERP definition and market reaction to ERP
implementation
announcements. Based on this review, the chapter will discuss the
constructs that
will be utilized in the research model, which in turn is presented in
the
following section. This part also includes the hypothesis. After this,
the
research model and design will be argued for. Thereafter there will be
an
analysis of the primary data results. In the following section, the
research
will be refined, results discussed, and a discussion of the research
and
limitations to the chapter which incorporates issues related to future
research
within the field. Thereafter the chapter will end with summary and
conclusions
derived from the research in this chapter.
Literature Review
In this section we will give a brief introduction to ERP systems and
thereafter present theory, regarding market reaction to ERP
implementation announcements, followed with hypothesis derived from the
different theory parts considering ERP implementation announcements,
such as contextual factors.
ERP Background and Definition
An ERP system is a system that allows companies to integrate various
departmental information. For many users an ERP system is a “does it
all” system, meaning that it performs everything from entry of sales to
customer service (Gupta 2000). The shortcomings of previous system like
Manufacturing
Resource Planning (MRP1) and Material Resource Planning (MRP2) led to a
development of a total integrated solution called enterprise-wide
resource planning, which attempts to integrate the suppliers and
customers with the manufacturing environment of the organization. Read
(1998) points out that ERP has three key components to assist the
management of manufacturing processes: client-server; real time
planning and work flow. Most of the ERP systems exploit the power of
this three-tier client server architecture. In a client server
environment, the server stores the data, maintaining their integrity
and consistency and processes the requests of the user from the client
desktops. The load of data processing and application logic is divided
between the server and the client. The three-tier architecture adds a
middle stratum, embodying all application logic and the business rules
that are not part of the application, enforcing appropriate validation
checks. Many of the companies implementing ERP systems has possibly
multiple locations and control. Hence the transfer of transactions or
online data has to be done across multiple locations. In order to
facilitate these transactions, ERP has to enable other important
technologies such as Workflow, Workgroup, GroupWare, electronic data
interchange (EDI), Internet, Intranet, data warehousing and so on and
so forth.
According to Gupta (2000), the supply chain capabilities of an
ERP system increase efficiency and productivity for their users. With
the help of linking supply-chain applications with other business
systems, users can slash cycle time times and reduce inventory. ERP
also allocates the organization to better connect with its suppliers,
distributors and end customers, which benefits the organization in
sharing information with their suppliers and customers. Assessing and
delivering information in real time helps the companies to better react
to their customers’ needs.
Market Reaction to ERP Implementation Announcements
Some researchers like Dos Santos et al. 1993; Hayes et al 2000; Tam
1998 have begun studying the market’s reaction to information
technology (IT) investments being made by firms. Determining whether
investments in information technology (IT) has an impact on firm
performance has been an major problem for information systems
researchers and practitioners. Financial theory suggests that managers
should make investment decisions that maximize the value of the firm
(Dos Santos et al, 1993). Dos Santos et al conducted a cross sectional
analysis that revealed that the markets react differently to
announcements of innovative IT investments than to follow up or
noninnovative investments in IT, meaning that innovative IT investments
increase firm value , while noninnovative IT investments do not. On the
other hand, as IT has become an increasingly important part of today’s
business, there has been an advocate to understand the relationship
between the use of IT and its impact on firm value (Hayes et al, 2001).
Dos Santos et al. (1993) and Tam (1998) found equivocal results
concerning the association between IT investments and market returns.
Hayes et al. (2000) indicated a mixed market reaction to the
outsourcing of IT functions. These researchers suggest that the market
considers the type of IT investment and contextual factors when
reacting to IT-related firm announcements, and that future research
should attend to these factors. Accordingly, we conduct our research as
Hayes et al. (2001). We investigate the market’s reaction to firm
announcements reflecting a specific type of IT investment (ERP systems)
and examine two salient contextual factors (firm size and health)
In that context, previous research cinducted by Hayes et al
(2001), indicates that there is a positive market return on IT
investments. Since these studies have been conducted most likely in the
US, the authors of this chapter think it is
interesting to look into the Norwegian market to find similarities. If
there is a difference between the US or Europe and Norway, further
research should be conducted to find reason. Our interpretation is that
it is most likely the same in the Norwegian market. There might be
differences in how the market sees firm size, firm health that is two
salient contextual factors and vendor selected. Since our research is
based on a short schedule, our findings might be implications and
future research in form of thesis’ can examine to a bigger extent the
reality.
Overall Market Reaction
As Hayes et al (2001) states in their article, the initial financial
investment made during ERP implementation will be recorded in the
firm’s financial statements as expenses and asset acquisitions. Such
expenditures are also reflected in the statement of cash flow as
decreases to cash. Without additional information, the market may view
the expenditures as relatively transitory in nature, without the
ability to impact future net cash flows of the firm. Furthermore, with
no information other than changes in the financial statements, the
market is unable to determine if the expenditures have the ability to
impact future cash inflows, and as a result, the firm’s market value
may not be significantly affected by the ERP implementation decision.
Moreover, if the market is provided with information concerning the
nature of the expenditures, investors have a better opportunity to
assess the impact of the costs future net cash flows. The importance of
providing nonfinancial forward-looking information of this nature in
external reports has been receiving increasing attention by the
accounting profession. Accounting research in this area indicates that
strategically oriented, nonfinancial information is associated with the
market valuation of the firm (Amir and Lev 1996; Hughes 2000).
Following Hayes et al (2001) arguments, by disclosing its
decision to implement an ERP system, a firm is providing the market
with information that cost increases are primarily transitory in
nature. That is, once the installation of ERP has been accomplished,
the increase in costs will not persist. Therefore the market should not
view the current increase in costs as having a permanent or long-term
negative effect on the future cash flows on the firm. The
implementation costs associated with ERP should result in long-term
benefits such as improved productivity and profitability, which can
have permanent effect on future cash flows. It is anticipated that the
market will take the information provided by the ERP announcement,
quickly and efficiently impound such information, and change market
values to reflect the expected net cash flow adjustments. While it is
expected that, on average basis, the market will consider the
discounted value of long-term benefits associated with ERP systems to
be greater than the transitory costs associated with implementation it
is possible that the market may negatively value the ERP
implementation. Girard and Farmer (1999), arguments that the market may
view ERP implementation negatively if it believes that the
implementation costs of ERP are so great that the firm cannot survive
implementation, or that implementation is so hazardous that market
share is adversely affected (Reuters 1999). Therefore our assumption
related to markets view positively on implementation announcement,
might fall behind when there are situations where a negative
association may result.
Contextual Factors
There are a variety of contextual factors that may help explain the
direction of the market’s reaction. As Hayes et al states in their
article, prior research has shown that the market reacts more strongly
to information provided by small vs. large firms (e.g.,Atiase 1985;
Feroz and Wilson 1992; Grant 1980; Hayes et al 2000).. The reason is
that the media and analysts follow large firms more closely and
therefore more is known about large firms in the capital market. The
result is that the incremental value of large-firm announcements is
diminished when compared to small-firm announcements since there is a
lower level of knowledge in the market concerning small firms, which
have less media attention and fewer analyst followings. Another factor
that may influence the strength and direction of the market’s reaction
to small-firm announcements is the financial health of the firm, since
prior research reveals that the market reacts differentially to healthy
and unhealthy firms (Hayn 1995; Khurana and Lippincott 2000).
While ERP implementation costs are viewed as short-term outlays, it can
take anywhere from six months to two years to implement the whole
system, thus expenditures or costs can span a considerable period of
time (Cooke and Peterson 1998). As Koch (1996) argues, a firm might
face significantly large cost overruns which have not been uncommon for
ERP implementations projects due to project schedule overtime and etc.
Additionally, during the implementation period, a firm may actually see
a decrease in performance before full ERP benefits can be realized (Wah, 2000). As a result it is estimated that it can
take two or five years to achieve a positive payback or significant
return on the ERP investment (Davenport 2000; Stedman 1999; Wah 2000).
If firms are to withstand the resource strain of ERP implementation
costs and subsequent operating expenditures without realizing immediate
financial benefits, they must have access to sufficient resources to
sustain them through this stressful period. For the market, the
resource strain of ERP implementation may be particularly important
with regard to small firms.
When compared to large firms, small firms are considered to be
more risky, as they tend to be experience losses more frequently (Hayn,
1995) and have greater variability in liquidity and solvency
measures (Huff et al, 1999). Hayes et al (2001) argues that, there
would be more risk associated with the ability of small/unhealthy firms
to remain financially viable through a prolonged period of resource
drain when compared to small/healthy, large/healthy, and
large/unhealthy firms. The primary reasoning is that small/unhealthy
firms are less likely to have sufficient resources or credit to
withstand a sustained resource drain of this nature. However,
considerably less uncertainty of this nature exists for the other
aforementioned firm categories. On the other hand, while small/healthy
firms may be viewed as having access to adequate resources, the
comparative burden of ERP implementation is not expected to be as great
for large firms, regardless of their financial health, due to the
relatively smaller portion of total overall costs consumed by the
implementation. Firm size is assessed by dividing firms
into small and large based on median total assets, accordingly to
Grødem
(2004), large Norwegian firm is a Small / Medium US company. There is
no standard definition of a firm’s financial health, but on the other
hand, Altman’s Z-score is one measure that has been used in assessing a
firm’s financial health (Barron et al. 1999; Miller and Skinner 1998;
Newberry and Dhaliwal 1998). According to Altman, a Z-score greater
than 2.99 indicates a financially healthy company (Wheelen and Hunger
1986). The formula for calculating the Z-score is (Hayes et al.
2000).
Z=1.2(WC/TA)+1.4(RE/TA)+3.3(EBIT/TA)+0,6(MVE/TD)+1.0(Sales/TA)
Where:
Z=indicator of a firm’s financial health.
WC/TA= working capital divided by total assets. (Arbeidskapital/sum
eiendeler)
RE/TA= retained earnings divided by total assets (annen
egenkapital-/sum eiendeler)
EBIT/TA= earnings before interest and taxes divided by total assets
(inntekt før skatt/totale eiendeler)ebit
MVE/TD= market value of equity divided by total debt,(markedsverdi av
aksjer-egenkapital/gjeld and
Sales/TA= sales divided by total assets. (salg/totale eiendeler)
Vendor Effects
We will also base our analysis on how the market reaction to ERP
announcements is based on the ERP vendor. While previous research has
no evidence indicating that the market reaction to ERP announcements
will differ based on the vendor, there is corollary evidence that the
market reacts more positively and strongly to higher quality auditors
(Balvers et al. 1988; Beatty 1989; Teoh and Wong 1993). In the
Norwegian market we assume that SAP and Microsoft are the biggest
vendors and most recognized for large firms, while in the small and
medium enterprise market Visma and Mamut is the most common used.
Additional supporting evidence suggests that quality of underwriters
has a positive impact on the offer price of initial public offerings
(Hanley, 1993). Perceived quality differences
for audit firms and underwriting firms are highly correlated with firm
size. In our study we don’t suggest that the largest ERP vendors offer
higher quality products than smaller vendors, but based on previous
research, we do expect a more positive market reaction if the
announcement is associated with a larger, as compared to smaller ERP
vendors. For purposes of the study, ERP vendor size is assessed using
latest year’s sales (as of 2003) and earnings before interest and taxes
(EBITDA) as indicators. SAP ( 7 billion euro sales and 1,8 million euro
EBITDA) and Adviso/Peoplesoft (408 million NOK sales and 73 million NOK
EBITDA) are the largest ERP vendors in the sample.
Underpinning the discussion of theory, presented in section 1, and the arguments put forward in this section, the following research model is presented.
This research model is built with the construct “A rich and B
limited.” Market reaction to ERP implementation announcements is the
dependent construct in the model and is defined by how the market
reacts to a firm publicly announcing an ERP implementation. The market
reaction can be seen as how the market sees the implementation
announcements and value the firm. The valuation of the firm is based on
the stock price, as given on Oslo Bors.
The independent constructs contextual factors and Vendor effects are
derived from a previous research conducted by Hayes et al (2001).The
construct contextual factors implies that the market will react
differently to firm size and financial health of the ERP implementation
announcer. As described in the theory, the rationale is that the market
will value different among small/healthy and large/unhealthy than
small/unhealthy and large/unhealthy (Hayes et al. 2001).
The other independent construct Vendor effects imply that markets
will react differently by the size of the ERP vendor. The rationale for
this construct is the prior research conducted by Hayes et al (2001),
which revealed the reaction to announcements involving large ERP
vendors was more positive than for small ERP vendors.
As described in theory, the anticipation of an interaction between firm size and health will lead the markets to react more favorably to ERP implementation announcements from small/healthy firms and large unhealthy firms, as both types of firms will be able to endure the implementation costs. Furthermore, the strong market reaction to small/healthy firms is influenced by the incremental value of the ERP announcement due to firm size and the firm’s ability to gain from increased growth and competitive positioning (Cabral 1995). For the large unhealthy firms the market is able to look beyond a short-term financial position, as Hayn (1995) suggests that firm losses may be viewed as temporary by the market, and evaluate the firm’s potential to become more efficient, profitable and competitive. In light of the above mentioned, following hypothesis is presented
H1: Firm size and health will interact such that the
market reaction to ERP announcements will be more positive for
small/healthy and large/unhealthy firms
Moreover, the market reaction to small/unhealthy firms will represent the other end of the spectrum, particularly because their ability to endure the costs of an ERP implementation is in doubt. Finally, the market reaction to large/healthy firms is expected to fall between the extremes, since the incremental information content of the announcement will be relatively low due to firm size, and the assessment of non-financial information becomes less important since the firm is already healthy. On the other hand, it is unclear that ERP implementation will cause a large/healthy firm to substantially increase competitive position or profitability relative to small/healthy and large/unhealthy firms. Thus, an alternate hypothesis is presented to look at the other side of the spectrum regarding H1.
H2: Firm size and health will interact such that the market
reaction to ERP announcements will be more negative for small/unhealthy
and large/healthy.
Thus, following the lead from above arguments in the theory part
that the
size of Vendor will direct the amrket reaction to ERP implementation
announcements, a final hypothesis is presented.
H3: The market reaction to the announcements of ERP implementation will be more positive for large ERP vendors (SAP and Peoplesoft) as opposed to small ERP vendors (all others)
Research Design
We choose a quantitative method in order to find facts and
causality. The reason for such choice is that our theory hardly can be
measured by a qualitative approach since data are based on statistical
evidence conducted from Hegnar.no and oslobors.no. The use of
quantitative method gives the authors the opportunity to have a large
sample size, the analysis will be based on facts as in statistical
evidence and summarizations. The use of secondary data found in annual
reports and on the net makes a base for the research in this chapter.
Data concerning the financial health of a firm is collected from
annual reports from the implementation announcement year. This data
will then been put into a spreadsheet and analyzed regarding the
z-score formula. Data concerning the market value of announcing firms
is collected from Hegnar.no. The rationale and choice of choosing
secondary data as sources for the analysis, is that it is the most
appropriate way in this research. The use of in-depth interviews would
give an indication of how analysts percept markets reaction to ERP
implementation regarding Vendor size and Contextual factors. The use of
secondary data as facts will give a more comprehensive picture of the
reality.
Sam
ERP implementation announcements were ascertained from various instances such as Google. no and press releases from both the implementing firms and Vendors. Firms publicly announcing an ERP implementation from 14.12.2000 through 25.10.2004 were initially included in the sample. A keyword search was employed using a combination of the search terms “Implement”, “Contract” and “Convert” with the name of each of the following ERP vendors: SAP, IFS Application, Peoplesoft, Oracle, Ibistic and Agresso. These search terms yielded an initial sample of 35 announcements. Subsequent review of the sample revealed 20 announcements of non-value due to missing dates and missing company information. The deletion of announcements led to 15 announcements with consistent information. Announcements with no confirmation date, duplicate press releases (such as contracts, etc) and missing financial data were excluded from the initial sample, leaving an appropriate sample of 15 announcements left, as reflected in table 1.
| Table 1: ERP Vendors and the Number of announcements associated with each Vendor | ||||
|
|
||||
| SAP | 6 | |||
| IFS | 3 | |||
| Peoplesoft | 2 | |||
| Oracle | 1 | |||
| Ibistic | 2 | |||
| Agresso | 1 | |||
| Total | 15 | |||
Table 2: The Number of ERP Vendors associated with Size and Health Sample Firms
|
|
Size of sample firms | Financial Health of sample firms | |||
|
|
Small | Large | Unhealthy | Healthy | |
| Vendor | |||||
|
|
|||||
| SAP | 4 | 2 | 2 | 4 | |
| IFS | 3 | 0 | 3 | 0 | |
| Peoplesoft | 1 | 1 | 1 | 1 | |
| Oracle | 1 | 0 | 0 | 1 | |
| Ibistic | 0 | 2 | 1 | 1 | |
| agresso | 0 | 1 | 1 | 0 | |
|
|
|||||
| Total | 9 | 6 | 8 | 7 |
In table 2, we have decided to divide firms into large and small based on median total assets. Firms are healthy if the z-score are greater than 2, 00 which were the median in the sample. All other firms are considerer unhealthy. .
The analysis of the sample are based on the average increase in stock price in given windows. In initial testing, three day event window were used.day -1, day and day 1. Additonally a 40 day window were used: day -20, day and day 20. There were some differences in between these two windows for the sample initiated. Hovewer the results indicated an overall positive market reaction to ERP implementation announcements.
The first (H!) and second (H2) hypothesis' identifies contextual factors that may impact the markets assessment of an ERP implementation announcement. Specifically, H1 and H2 predicts an interaction between firm size and health. Firm size is measured by dividing the sample into large and small firms using the median of total assets. There is no standard definiton for a firms financial health; however , Altmans z-score is one measurement that has been used un assessing a firm's financial health(ref). Additonally, the z-score indicating a firm's financial health were adjusted by the median in the sample, which was 2,0. Hence the sample is divided into firms with Z-scores>2,0 (healthy) and <2,0 (unhealthy). To test for the expected interaction between firm size and health, the average increase in stock price is used by interpreting the numbers. Additionally the numbers were interpeted from both the 3 day event window and the 40-day event window.
Finally, the third hypothesis (H3) posits a large/small vendor effect. Hypothesis 3 is tested by dividing the sample into large ERP vendors (SAP and Peoplesoft) and small vendors (all others).
As shown on table, the mean and average increase in stock price were postive during the three day event window and the 40 day event window. It also appears that the firms in the sample were equally distributed among healthy and unhealthy firms according the 2.0 benchmark.
Table 3: Descriptive statictiscs for the Event Window Applied
| Mean | Std.dev | Minimum | Median | Maximum | |
| 3-day window | 0,02% | 1,61% | -3,92% | 0,43% | 2,46% |
| 40-day window | 7,30% | 5,14% | 0% | 6,49% | 19,72% |
| Total assets (in million NOK) | 709933 | 129274,7 | 1503 | 186302.5 | 4826173 |
| Z-score | 2,03 | 0,895 | 0,5 | 2,00 | 3,77 |
Results for the overall market reaction are shown in table 3. The mean standardized average increase in stock price were positive and indicates that there was a slightly positive market reaction in the three day event windowm while the 40 day event window show a slightly more significantly positive market reaction to ERP implementation announcements. Hence, the overall market reaction to ERP implementation announcements are postive, as the total average increase in the tested windows indicates.
With positive market value
in the period, one can argue that the stock followed the OBX (Oslo
børs
hovedindeks). Taking this argument into confinement, we thought there
was a need
to analyze our findings regarding stock value and compare them with the
OBX in
the same period.
To test the first hypothesis and the second hypothesis, the mean average increase in stock price during the two windows. The 3 day event window shows that the average increase in average stock price were largest for small/healthy and large unhealthy firms, hence the first hypothesis is somewhat supported. By looking at the 40 day event window there is a slightly higher increase for large healthy firms than for large unhealthy firms which in turn raise some doubts about the first hypothesis. The reason for this will be explained in further detail in the discussion part of this chapter. Additionally, the mean average increase in stock price for small7unhealthy and large unhealthy firms are more negative than for small/unhealthy and large/unhealthy. Hence, the second hypothesis is supported in the three day event window. Further interpretation of the data in the 40 day event window indicates as mentioned before that the mean average increase in stock price for large/healthy firms are slightly larger than for large/unhealthy firms. Hence, hypothesis two is to some extent supported. Overall both Hypothesis one and two are partially supported by the evidence.The test statistics for hypothesis one and two are shown in table 4
Table 4: Mean statistics for the inteaction between firm size and financial health of the firm
|
|
mean 3-day | mean 40 day |
| small/healthy | 0,62 % | 11,02 % |
| small/unhealthy | 0,36 % | 4,79 % |
| large/healthy | -0,14 % | 7,96 % |
| large/unhealthy | 0,49 % | 7,90 % |
The testing of hypothesis 3 (H3) is shown in table 5 The market reacted more positive to SAP in the three day event window than the smaller vendors such as IFS, Ibistic and Agresso. The negative market reaction to Peoplesoft can be explained by the relatively small sample. On the other hand, by looking at the mean average increase in stock price during the 40 day event window it seems like there arent any differences among the Vendors. Considering the skew announcements associated with the listed vendors there are some indications that large ERP vendors affects the markets raction to ERP implementation announcements. Hence, hypothesis three are somewhat supported.
Table 5: Mean statistics for the average increase in stock prices associated with size of ERP Vendor
| Vendor | Sales | Ebitda | Mean in % 3-day | Mean in % 40-day |
| SAP | 7025 mill euro | 1783 mill euro | 0,44 % | 6,67 % |
| IFS | 262 mill nok | 21,7 mill nok | -0,69 % | 2,60 % |
| Adviso/peoplesoft | 408 mill nok | 73 mill nok | -1,26 % | 6,21 % |
| Oracle | usd 9,5 mrd | 0,90 % | 19,72 % | |
| Ibistic | 24,3 mill nok | 1,6 mill nok | 0,12 % | 12,50 % |
| Agresso | 137 mill nok | 15 mill nok | 0,00 % | 1,52 % |
(The numbers for SAP and Oracle are published on a total world basis, and most of Oracle earnings are derived from sales of databases)
As we have found indiactions that supports our hypothesis to some extent, the proposed research model is kept. The rationale off the choice is that this study didn't reveal any new constructs, although additional constructs are discussed later in the chapter.
The authors of this chapter considers that the main weaknesses in this chapter relies heavily on the relatively small sample size related to ERP implementation announcements. After sondering the net after press releases and similarities concerning ERP implementation announcements one may derive that managers do not perceive announcements as important. Insufficient information related to the ERP implementation, missing finacial data and contradictary press releases such as contracts and etc led to the releatively small sample. Another important issue might be the weakness of the statistical testing of the sample. Since the numbers are based on average increase in stock prices, the conclusion derived are indications on the reality and further testing for significance should be applied. One common technique used in prior research is event-time methodology, in which the hypothesis were tested by using abnormal market returns. Abnormal market returns measure the difference between actual returns for a given time and the expected return at the time.
Moreover, concerning the calcualations there would be beneficial to introduce Betas, also known as as beta coefficients or beta factors, which are a popular analytical tool relied on by investors, portfolio managers, and market analysts to measure the volatility or risk of stocks. A beta measures the extent to which the price of a given stock varies with respect to the market as a whole. Using these coefficients to normalize the movement of stocks, will prevent the biased results (Burkay. 2004). Finally, checking the net sellers and the net buyers during the rise and the fall of the stock would be extremely helpful in order to justify the findings. This might also be useful whether the company sells and buys its own stocks with a kind of insider information-
On the other hand, future research in this area might also take into consideration other factors such as what kind of partner used, eloborating on the fact that there aren't many pure ERP vendors in Norway. Most of the available solutions on the Norwegian market are offered through various partners and consultancy companies providing help in the implementation phases. Additionally, there are several vendors mssing from the sample which could have been beneficial in the findings.
In the end the results are therefore to some extent vague concerning the hypothesis testing. Although, the overall market reaction indicates a positive market reaction to ERP implementation announcements further investigation and statistical testing of the hypothesis' should be employed to find a significance.
The purpose of the current study is to examine to what extent the reaction of the capital market to firm announcements concerning ERP implementation plans. As Hayes et al (2001) revealed, ERP vendors and advocates have provided many reasons to expect substantial net benefits arising from ERP implementation announcements, as given in increase of stock price, there is no empirical evidence in this regard. Furthermore, the testing has been procedurer in the Norwegian market which have never been done before. Hence, this study is unique as no other accounting or information systems research has been conducted in this field among Norwegian firms listed on Oslo Stock Exchange.
In the current study, the total average increase in stock price during both a three day event window and a 40 day event window for fims publicly announcing an ERP implementation. All ERP implementation announcements from date to date were initially included in the sample. Further investigation of the sample led to a comprehensive sample of 15 announcements with sufficient information and financial data.
Overall, the market responded positively to ERP implementation announcements. Further analysis indicated that the market reacted to some extent differentially depending on the size and health of the announcing firm, as a total average increase in stock price were analyzed. Specifically, the market responded most favourably to small/healthy and large/unhealthy in the three day event window applied. Although, there were differences analyzing the 40 day event window, there is a slightly indication of markets reacting more positively to small/healthy and large/unhealthy firms. Moreover, the market reacted most positively to small/healthy firms and less positively to small/unhealthy firms. The reaction to announcements by large/healthy and large/unhealthy firms differed among the approaches applied.In instance, in the three day event window, markets reacted positively to large/unhealthy firms and negatively to large/healthy firms. On the other hand, applying the 40 day evet window the situation turns around with most positively reaction to announcements by large/healthy firms and less positively to large/unhealthy firms. Deriving from this, there are some uncertainty on whether firm size and financial health interacts. In that context there are to some point an interaction, although the results vary among the approaches applied in the study.
Additionally, size of the ERP vendor impacted the market reaction to ERP implementation announcements. The reaction to the announcements involving large ERP vendors, as reflected in SAP and Peoplesoft varied. Although a variation in firms average increase in stock price associated with each vendor, there is some implications that the market reaction are slightly more positive for large ERP vendors as opposed to small.
In the overall context, the empirical evidence collected in this study reveals that investors perceived ERP implementation announcements as "good" news.for most firms, as reflected in a positive total average increase in sock price. This findings are important to firm managers since they are responsible for simultaneously improving profitability and market capitalization (Hayes et al. 2001). While this study does not measure the extent to which ERP systems actually improve the financial health of adopting firms, it provides some indication that the marketexpects positive net future cash flows from the use of ERP systems for all in the 40 day event window, and except large/healthy firms in the three day event window. The reaction of investors to ERP implementation plans also highlights the importance of providing non-financial forward-looking information of this nature in external accounting reports, as the study results indicate that the provision of strategically oriented, non-financial information can impact the market value of the firm.
Enterprise Resource Planning systems (ERP) have been presented as the panacea for today’s businesses, expected to deliver dramatic efficiency and effectiveness gains. By 1999 over 60 percent of Fortune 1000 companies had implemented ERP systems (Piturro 1999; Stein 1999). Properly implemented, these systems can significantly improve operational efficiency and effectiveness. However, research shows that many organizations are disappointed with the results, frustrated by the lack of expected business benefits (Al-Mashari 2003). For example, a survey made by the PA Consulting Group (2000) including over 50 pan-European organizations, revealed that 92 percent of over 50 senior directors surveyed were dissatisfied with results achieved to date. The PA Consulting Group’s survey is only one of many studies that reveal a discrepancy between companies’ initial expectations for their ERP systems and the actual accomplishments (Al-Mashari 2003).
When actual accomplishments fall short of expectations, the discrepancy results in dissatisfaction (Prenhall 2004; Staples et al 2002; Mahmood et al 2000). This discrepancy may also be described as disconfirmation. This occurs when expected performance and actual performance differ from one another; negative disconfirmation occurs when actual performance is less than expected performance, and positive disconfirmation occurs when actual performance exceeds expected performance (Szajna and Scamell 1993). One can clearly see the rationale behind Staples et al’s (2002) argument that unrealistically high expectations are associated with lower levels of satisfaction and thus, maintaining expectations at a realistic level would be in management’s best interests. However, studies have also revealed that unrealistically low expectations correspond with lower levels of satisfaction; “…those who have experienced positive disconfirmation of their expectations will have perceptions that will be assimilated toward their expectations and will be less than those with realistic expectations” (Szajna and Scamell 1993, p. 495). Therefore, maintaining expectations at a realistic level is deemed important for satisfaction and considered to be relevant for managers.
A useful term for building the argument that expectations influence satisfaction is derived from cognitive sciences, where ‘mental models’ are forwarded as mechanisms that enable humans to describe systems and explain system functionality (Rouse and Morris 1986). Reichart (2003) has further claimed that mental models serve as the basis for our expectations. In other words, there is a relationship between system descriptions and expectations. It is therefore reasonable to infer that a company’s description of their IT system is the source of the expectations towards the system. The relevance of this discussion lies in regarding expectations as pre-implementation belief, given by a description of a given ERP system in terms of its expected functionality. Satisfaction is considered as a post-implementation experience, which is influenced by expectations by virtue of the mentioned cognitive construct, and that is given by a description of the satisfaction with perceived system functionality.
Having pointed out the discrepancy between expectations and actual performance, the authors of this chapter argue that this discrepancy may influence overall satisfaction for an ERP customer (i.e. the company that purchases and implements an ERP system). As satisfied customers, i.e. ERP system buyers, are more valuable to the firm (Fojtik 2002), i.e. ERP system vendors, companies should meet customer expectations in order to attain satisfied customers (Kotler 2003; Levitt 1960). However, as many ERP ventures involve substantial business process analysis, employee retraining and altered work procedures, a lack of understanding of the complexity of the implementation process is argued to be a factor influencing this discrepancy, affecting satisfaction with the ERP implementation (Adam and O’Doherty 2000; Al-Mashari 2003; Davenport 1998).
The objective of this chapter is to investigate whether the discrepancy between pre-implementation expectations of ERP effects and post-implementation experience has implications for customer satisfaction, i.e. a company’s satisfaction with its ERP system. The following research question is proposed:
“Will more realistic expectations of ERP effects lead to raised customer satisfaction regarding these systems?”
The chapter will now continue with a presentation of the theoretical backgrounds for ERP effects, customer expectation and satisfaction. Based on this review, the chapter will present the constructs that will be utilized in the research model, which in turn is discussed in the following section. After this the research model and the hypothesis are presented. Thereafter, the research design and method are discussed. This is followed by an analysis of the primary data results. In the following section, the research model will be refined, practical implications discussed, and theoretical implications and suggestions for future research forwarded. Before the chapter ends with conclusions, research limitations will be outlined.
In order to investigate the proposed research question, the theoretical backgrounds for ERP effects and customer satisfaction are presented in this section. The aim is to elaborate on the theory. The following passages introduce the concepts of ERP effects and customer satisfaction and discuss the constructs that will be utilized in the research model.
System integration has been identified as one of the central benefits of ERP systems and also one of the main reasons that companies choose to implement ERP systems (Davenport 1998; Adam and O’Doherty 2000; Barker and Frolick 2003; PA Consulting 2000). An example is Eastman Kodak Co.’s decision to completely redesign their business around an integrated system and a common set of business processes, facilitated by an ERP system. Prior to this reorganization, they were maintaining a pool of 2600 fragmented software applications, over 4000 system interfaces, and roughly 100 programming languages (Stevens 1997). The resulting fragmentation of systems and information was seen as an obstacle, identifying the need for integration. Davenport (1998) states that such fragmentation of information across computer systems represents one of the greatest inhibitors for organizational performance and productivity because of the maintenance costs associated with the lack of enterprise-wide integration. An explanation for why organizations have become so cluttered with un-integrated systems may be that different systems have been introduced along with the growth of firms, and that compatibility may not have been a main focus. As time progressed, the systems would become more and more mismatched, making the need for an integrated solution more visible (Barker and Frolick 2003).
The main advantage of an ERP system is that it secures benefits and value to the organization by integrating key business processes such as finance, manufacturing, sales, human resources, logistics, etc. (Adam and O’Doherty 2000; Bajwa et al 2004; Davenport 1998). Furthermore, an ERP system ensures a common data pool used across the system (Adam and O’Doherty 2000; Davenport 1998) and, in addition to removing the need for fragmented end-user tools, it standardizes operating procedures and reporting (Adam and O’Doherty 2000), and improves responsiveness (Al-Mashari 2004).
Given their potential, ERP systems may seem to be dreams come true for managers. However, the process of transforming a fragmented organization with fragmented IS to a fully integrated organization based on ERP is not easy. ERP implementations are extremely complex (Adam and O’Doherty 2000; Al-Mashari 2003; Davenport 1998). They require vast amount of human and financial resources and an appropriate degree of process re-engineering (Adam and O’Doherty 2000), as the ERP eventually demands a business to adopt a process-based organizational structure (Al-Mashari 2004; Nah et al 2003). Many organizations have experienced ERP project failures along these requirements (Al-Mashari, 2003; Bajwa et al 2004; Barker and Frolick 2003), and some have even led companies to bankruptcy (Davenport 1998). Caldas and Wood (1998) argue that ERP is hype, and that many companies engage in ERP implementation projects as a result of over-promotion of the system’s capabilities. Caldas and Wood (1998) further claim that the link between ERP and competitive advantage is difficult to ascertain. On the other hand, Barker and Frolick (2003) argue that the negative effects of ERP are few and that the ones that do exist are related to the changes an implementation of an ERP system brings to the business processes and practices, not the actual software installation. Hence, an ERP system implementation needs to be complemented with appropriate organizational changes (Pawlowski et al 1999). The absence of adequate change management can easily result in a total failure of the ERP initiative (Bancroft et al 1998).
There are other reasons why ERP implementation efforts fail to offer expected benefits. Too narrow focus on technical aspects (Al-Mashari 2003; Davenport 1998; McNish 2002) has been forwarded as a reason; another is the factor of urgency, leading to poorly planned projects (Caldas and Wood 1998). Researchers argue that many companies embarking on ERP projects do not understand the scope of the ERP project, and therefore do not allocate enough resources (Al-Mashari 2003).
The fact that ERP implementations fail to provide expected benefits influences customer satisfaction with these systems (Staples et al 2002; Mahmood et al 2000). The pursuit of customer satisfaction is grounded in the belief that satisfied customers are more valuable to the firm because they are expected to: be retained longer and in greater numbers, buy more goods, cost less to serve, be willing to pay slightly higher prices, respond faster to promotional efforts, suggest and evaluate new products and revenue streams, and refer others - which helps reduce the cost of acquiring new customers (Fojtik 2002).
Today customers are faced with a vast array of products, brands, suppliers and prices. According to Kotler, customers make their choices based on their perception of what alternative will deliver the most value (Kotler 2003). He argues that customers are value-maximizers, limited within the bounds of search cost, limited information and knowledge, and limited mobility and income. In line with the topic of this chapter, Kotler continues by saying that customers “form an expectation of value and act on it” (Kotler 2003, p. 60). Satisfaction is influenced by whether or not the offer meets customer expectations. Therefore, as expectations directly relate to the customer’s perception of value, by meeting expectations the customer will most probably regard the transaction as satisfactionary and worthwhile. However, if the company sets expectations too low, it will not attract enough buyers, even if the product will exceed the expectations and satisfy those who buy (Kotler 2003). Hence, “the successful marketer realizes that the smart strategy is to accurately portray the product and service attributes with the expectation that, at minimum, the product will satisfy [i.e. meet expectations], and with a little effort, even delight”(Singh 2004, p. 1).
Satisfying customer needs and wants by meeting their expectations has been a major goal of marketing for many decades (Kotler 2003; Levitt 1960). However, the Information Systems (IS) area of business does not have a reputation for meeting and exceeding customer expectations. Rather, IS projects are notorious for their high failure rate and many organizational experiences show that resulting outcomes of ERP implementations fall short of expectations (Al-Mashari 2003). The Meta Group estimates that half of all new US software projects go over budget (Hayes 1997). According to the Standish Group, 53 percent of IS projects overrun their schedules and budgets, 31 percent are cancelled, and only 16 percent are completed on time and on budget (Hayes 1997). Ambler (1999) found a 85 percent failure rate in the development of large-scale software projects. So, clearly the IS industry are struggling to meet customer expectations.
Szajna and Scamell (1993) propose that IS failure may be caused by the system’s inability to meet expectations. Furthermore, it has been proposed that unrealistic expectations of ERP effects are one of the reasons ERP system implementations fail (Umble and Umble 2002). This is explained by underestimations of the amount of time and resources that will be necessary for a successful implementation. Umble and Umble (2002), and Barker and Frolick (2003) argue that the key issues that need to be covered when planning a successful ERP implementation are the client’s expectations and the product’s capabilities.
Several studies have focused on the relationship between expectations about systems and satisfaction with the systems, and the findings suggested that realistic expectations lead to higher levels of satisfaction (Ginzberg 1981; Staples et al 2002; Szajna and Scamell 1993). Staples et al (2002) use disconfirmation theory to suggest that unrealistically high expectations are associated with lower levels of satisfaction and that maintaining expectations at a realistic level would be in management’s best interests. This proposition is relevant for the purpose of this chapter, however unrealistically low expectations also need to be discussed. Szajna and Scamell (1993) found that unrealistically low expectations correspond with lower levels of satisfaction. Their findings are supported by cognitive dissonance theory, which predicts that when actual performance exceeds expected performance, perceptions of performance (i.e. satisfaction) will be less than if expected and actual performance were more aligned (Szajna and Scamell 1993). In other words, if the systems capabilities exceed expectations, the level of satisfaction will be less than if the capabilities were closer to expectations. This view is further supported by Ginzberg (1981), who argues that realistic expectations influence higher levels of satisfaction. Consequently, both unrealistically high expectations and unrealistically low expectations have been found to cause lower customer satisfaction.
Based on the above discussion, the authors of this chapter argue that it is important to ensure that customers of ERP systems hold realistic expectations towards ERP effects. Therefore, by managing and altering customer expectations to become more realistic, i.e. to be more aligned with the actual effects, ERP vendors may positively impact customer satisfaction. As a result, the customer may expect certain negative effects such as time delays and budget overruns, and will thus not be as disappointed if they occur. Accurately portraying the ERP products attributes, i.e. ensuring that ERP customers hold realistic expectations of ERP effects, is therefore important for customer satisfaction with the ERP product.
In order to measure the degree to which a customer’s expectations of ERP effects are realistic, the authors propose a framework for investigating the discrepancy between expected and perceived ERP effects. The ERP Effects framework (Appendix 1: Table 1) proposed by the authors is adapted from Shang and Seddon (2000), who suggested five dimensions for evaluating ERP benefits after implementation of a system. These are Operational, Managerial, Strategic, IT infrastructure and Organizational. The ERP benefits framework (Shang and Seddon 2000) is incorporated into the proposed ERP Effects framework by regarding benefits as effects. Larsen (1999) has suggested that an evaluation of IS-effects needs to encompass more effects than the beneficial since benefits do not encompass all aspects of IS innovation. Therefore, the five dimensions proposed by Shang and Seddon (2000) are elaborated upon by supplementing them with additional findings concerning ERP effects in the literature. In addition, the Business dimension and the ERP project dimension have been added in order to provide a broader foundation for analysis. The following sections outline the ERP Effects framework by elaborating on the proposed dimensions: (1)Operational, (2)Managerial, (3)Strategic, (4)IT infrastructure, (5)Organizational, (6)Business and (7)ERP project. Table 1 (Appendix 1) shows a comprehensive summary of the dimensions.
Operational
Shang and Seddon (2000) define the Operational dimension in terms of
operational activities as day-to-day activities that are repeated
periodically. Since an ERP system supports business process integration
(Adam and O’Doherty 2000; Bajwa et al 2004; Davenport 1998),
standardized reporting (Adam and O’Doherty 2000), and improved
responsiveness (Al-Mashari 2004), it is expected to have an effect on
the organizational level in terms of cost reduction (Shang and Seddon
2000; PA Consulting 2000), cycle time reduction (Shang and Seddon
2000), productivity improvement (Shang and Seddon 2000; PA Consulting
2000; Davenport 1998), quality improvement (Shang and Seddon 2000; PA
Consulting 2000), customer service improvement (Shang and Seddon 2000;
PA Consulting 2000; Adam and O’Doherty 2000), responsiveness
(Al-Mashari 2004; PA Consulting 2000; Davenport 1998), standardization
of operating procedures and reporting (Adam and O’Doherty 2000; PA
Consulting 2000; Davenport 1998), and flexibility (PA Consulting 2000).
These effects are listed in the ERP Effects framework as points 1.a-1.g
in Table 1 (Appendix 1).
Managerial
The Managerial dimension encompasses such activities as allocation and
control of resources, monitoring operations and supporting strategic
decisions (Shang and Seddon 2000). ERP systems incur effects along the
Managerial dimension in terms of improved decision-making (Barker and
Frolick 2003; Shang and Seddon 2000). Better resource management has
also been suggested as a potential ERP effect (Shang and Seddon 2000;
PA Consulting 2000). Other expected effects are performance improvement
(Shang and Seddon 2000), management control (PA Consulting 2000;
Davenport 1998), information quality (Staples et al 2002) and
management system changes (Laudon and Laudon 2003). These are listed as
points 2.a-2.e in the ERP Effects framework (Appendix 1: Table 1).
Strategic
Another suggested dimension for evaluating ERP effects is the Strategic
dimension, that focuses on competitive advantage and differentiation,
and involves high-level business decisions such as marketing
competition, product planning and customer retention (Shang and Seddon
2000). Davenport (1998) has also emphasized the influence of ERP
systems on long-term, strategic benefits. Shang and Seddon (2000) have
suggested that ERP systems may potentially offer the following
strategic benefits: support business growth, support business alliance,
build business innovations, build cost-leadership, generate product
differentiation and build external linkages. These mentioned benefits
are incorporated into the proposed ERP Effects framework as effects
along the Strategic dimension (Appendix 1: Table
1).
Other effects, such as the ERP system acting as a catalyst for change
and contributing to improving supply chain management (PA Consulting
2000), have been proposed. Furthermore, ERP systems provide competitive
advantage (Barker and Frolick 2003; Kirchmer 1999) and help maintain
competitive advantage (Davenport 1998). These mentioned effects are
listed as points 3.a-3.i in the ERP Effects framework (Appendix 1: Table 1).
IT infrastructure
Investment in an ERP system represents a considerable initiative to
create an application infrastructure within the organization that
ensures a common data pool (Adam and O’Doherty 2000; Davenport 1998),
builds business flexibility for current and future changes, and
provides increased IT infrastructure capability and IT cost reduction
(Shang and Seddon 2000). PA Consulting Group (2000) has also proposed
that infrastructure effects include providing Euro handling and
prevention of the Y2K bug. The latter issue may still have relevance,
depending on when the implementation project was initiated. Another IT
infrastructure effect is that it removes the need for diverse end-user
tools (Adam and O’Doherty 2000). These effects are listed as points
4.a-4.e in the ERP Effects framework (Appendix 1: Table 1).
Organizational
The Organizational dimension of ERP effects is concerned with
organizational learning and execution of strategies (Shang and Seddon
2000). IT systems are often used to support organizational changes,
facilitate business learning and build common visions (Shang and Seddon
2000). In addition, employee empowerment through the use of ERP systems
has been suggested as a potential effect (Shang and Seddon 2000; Barker
and Frolick 2003). Integration has also been forwarded as an ERP effect
(PA Consulting 2000; Adam and O’Doherty 2000; Bajwa et al 2004;
Davenport 1998), linking the different units into a standardized,
interconnected and widely available application. As a result, ERP
systems may affect changes to the organizational structure (Al-Mashari
2003; Nah et al 2003; Pawlowski et al 1999). These effects are listed
in the ERP Effects framework as points 5.a-5.g in Table 1 (Appendix 1).
Business
Change management has been suggested as an important factor for a
successful ERP implementation (Al-Mashari 2003; Bancroft et al 1998)
because of the “multidimensional changes involved in ERP
implementations”
(Al-Mashari 2003, p. 41). The complexity of the implementation
initiative was also discussed by Davenport (1998), who stated that an
ERP system imposes a certain logical framework on a company, supporting
nearly all business activities. In this respect, an ERP implementation
entails business process changes (Davenport 1998; Adam and O’Doherty
2000; Pawlowski et al 1999) and business practices changes (Pawlowski
et al 1999). In addition, resistance to change as a result of IS
implementation is an important effect, influencing all employees; both
managers and line-employees (Laudon and Laudon 2003). This effect is
therefore included in the Business dimension of the ERP Effects
framework proposed in this chapter. The Business dimension effects are
listed as points 6.a-6.c in Table 1
(Appendix 1).
ERP Project
Throughout the implementation, the ERP project may be influenced by
changing objectives and customization (Davenport 1998). Adam and
O’Doherty (2000) found that the actual implementation of the ERP system
led managers to experiment with new functionality and objectives in
order to attain additional benefits. Allocation of resources has also
been proposed as an effect of the ERP project (Adam and O’Doherty
2000). In addition, Laudon and Laudon (2003) have proposed that project
duration is an influencing effect. The ERP project dimension effects
are listed as points 7.a-7.d in Table 1
(Appendix 1).
As discussed earlier, IS projects have a reputation of high failure rates, which is argued to be a result of unrealistic expectations. Such expectations toward product performance are formed by word-of-mouth from family, friends and colleagues, from media and marketers’ and competitors’ information (Kotler 2003). Kotler (2003) further proposes that customers are limited in their knowledge, first forming an expectation of value and then acting on it. In other words, perception of ERP system effects is influenced by expectations. Since customer satisfaction is defined as “the extent to which a product’s perceived performance matches a buyer’s expectations” (Prenhall 2004: [Online]), it is relevant to examine whether expectations influence satisfaction. The proposition is to evaluate customer satisfaction with ERP effects.
In order to operationalize customer satisfaction and retain relevance to the proposed area of study, customer satisfaction is measured along the same dimensions that were outlined in the previous section: Operational, Managerial, Strategic, IT infrastructure, Organizational, Business and ERP project. The proposed dimensions for evaluating the discrepancy between expected and perceived ERP effects will be used to assess customer satisfaction. The purpose is to evaluate customer satisfaction with the proposed ERP effects.
Based on the discussion in the previous section, the following research model is presented.
Figure 1. Research Model proposed by the authors of this chapter.
Realistic expectations are measured in terms of the discrepancy between expected and perceived ERP effects. The research model proposes that a higher degree of realistic expectations leads to a higher level of customer satisfaction.
The discussion presented in the previous section also gives the hypothesis of this study:
H1.: An ERP customer with more realistic expectations of ERP effects will report a higher level of customer satisfaction than an ERP customer whose expectations are unrealistic.
The approach chosen for this study is of a qualitative exploratory nature, underpinned by Lyytinen and Hirschheim’s (1987) argument that the study of IS failure, for general reasons, is a complex venture, reducing the possibility for producing accurate statistical generalizations. In line with this argument, a qualitative exploratory research design is considered suitable because it can provide significant insight, ideas and understanding about specific problems or opportunities. In addition it can help delimit the problem and clearly outline the information needed for future research (Easterby-Smith, Thorpe and Lowe 1991). Furthermore, as the term suggests, exploratory research is often conducted because a problem has not been clearly defined, or because its real scope is unclear (Easterby-Smith, Thorpe and Lowe 1991). Although established research has covered the relationship between realistic expectations of an IS system and satisfaction (Ginzberg 1981; Staples et al 2002; Szajna and Scamell 1993), little has been written about this relationship with regard to ERP implementations. Hence, this form of research design is therefore considered appropriate.
In exploratory studies, in-depth interviews can be very helpful to find out what is happening and to seek new insights (Robson 1993). According to Easterby-Smith et al (1991), an interview is undoubtedly the most advantageous approach to obtain data where the questions are complex, open-ended and where the order and logic of the questioning may need to be varied. Furthermore, semi-structured interviews are used in qualitative research in order to conduct exploratory discussions to reveal and understand the ‘what’ and the ‘how’ (Saunders et al 1997). The aim of this chapter is to examine whether more realistic customer expectations of ERP effects lead to raised customer satisfaction. Since the understanding of the ‘what’ and the ‘how’ (Saunders et al 1997), is explored through the interview guide and the analysis, this research strategy is considered appropriate.
The interviewees chosen are professionals within the ERP arena. They have both experienced and had responsibility for the implementation of an ERP system in their organizations, and thus are considered to contribute with valuable insights. It should be noted that names of companies and interviewees have not been given to preserve anonymity.
The interviewees are:
In the interview, the broad dimensions of the ERP Effects framework (Appendix 1: Table 1) were discussed in terms of pre-deployment expectations and post-implementation satisfaction. In order to ensure validity, the interviews were complimented with a follow-up questionnaire. This so-called ‘Between-method’ methodological triangulation (Denzin 1989) was used to improve validity of results. By combining the two methods of investigation, i.e. the interview and the questionnaire, the limitations of using a single-method of study were overcome by cross-checking sources. After the interviews, the participants were sent a questionnaire where all effects found through the literature (Appendix 1: Table 1) were taken into consideration. In the questionnaire the interviewees were asked to define if an effect was expected, if this particular effect fell-short, met or exceeded expectations, and if the interviewee was dissatisfied, satisfied of delighted with the particular effect. In addition, a Likert scale was used to rate the extent to which the outcome of a particular effect influenced the overall satisfaction with the ERP deployment (Appendix 2: Questionnaire). In order to facilitate systematic analysis, the interviews were transcribed and used jointly with audiotapes.
The outcomes of the interviews with the CIO and the RM were discussed and elaborated upon in the interview with the business consultant. This was considered useful for gaining practical understanding of managerial implications of the research results. In addition to the interview subjects mentioned above, the authors of this chapter interviewed a CEO of a consulting firm that specializes in ERP deployment. He will henceforth be referred to as the CEO, and his company as the Consulting Firm.
This part of the chapter will analyze the interviewees’ pre-deployment expectations of the ERP system effects and their post-implementation satisfaction. The two companies and their ERP projects will be presented in turn. Before the chapter embarks on an analysis of expectations versus satisfaction with ERP effects in relation to the research model, the current IT solution situation in the Shipping Company and the Oil Company, and sources of the need to change status quo will be presented. An analysis of the interviewee’s pre-deployment expectations will be presented into applicable dimensions of (1) Operational, (2) Managerial, (3) Strategic, (4) IT infrastructure, (5) Organizational, (6) Business, and (7) ERP project. Following this, there will be a brief description of the actual system which was chosen and implemented. The actual outcome in terms of ERP effects will be discussed in relation to the pre-deployment expectations. Thereafter the customer satisfaction in regard to the actual ERP effects will be analyzed. This part will also include relevant data collected through the questionnaire. This analysis of pre-deployment expectations and post-implementation satisfaction will lay as a base for the refinement of research model in the next section and also underpin the development of theoretical and practical conclusions at the end of this chapter.
The need for a more integrated approach to the Shipping Company’s IT solutions was created internally in the company. The CIO was the championing actor for this project, which was spurred by the notion that the status of the present IT solutions were lacking in functionality, standardization and interaction (where the Shipping Company had many differing types of IT solutions between the head office, overseas offices and vessels). Furthermore, several suppliers had terminated or wanted to end the ongoing support for the current IT solutions. Also, the technology the Shipping Company used was considered to be old-fashioned and not responsive nor flexible enough for the changing needs of the organization and the market. All this led to limited information access and control, difficulties to standardize and optimize business processes, and constrains to employ new business transaction opportunities like e-commerce with suppliers etc (CIO 2004).
In the acquisition phase a requirement review of business objectives of
the new IT solution was created. Relative to research model presented
in this chapter the Shipping Company’s explicit objectives are
categorized according to the following dimensions;
Managerial - To
enhance consistency in information and data, and
also improve access to information in order to support management
decision-making.
Organizational -
To increase cross-functional and departmental
integration and interaction, including integration between head office,
overseas offices and vessels; To enhance the possibility for
accomplishing organizational changes in the Shipping Company; To
enhance consistency in information and data and improve access to
information in order to enhance organizational skills; To increase the
organizational skills through increased insight regarding the use of
the Shipping Company’s IT-solutions.
IT Project - A demand for resource requirements for overall program management, for the Shipping Company’s involvement and training; To demand support for changing objectives through project change control, system operations model and key performance indicators.
An effect which was not included in the research model but in the CIO’s business objectives is the expectation that the new IT solution was required to improve support to IT-users, internally through the IT department refocusing towards business functionality, and externally through strong software vendor support. Although this is not an attribute to the IT solution as a technological artifact, rather to the augmented product of the system as such, this was an expected of the new IT solution venture. Moreover, the system was also expected to decrease employee turnover, which, again, is an effect not accounted for in the research model.
The ERP system that was chosen to meet the needs of the expected objectives was Peoplesoft EnterpriseOne, which is a suite of modular, pre-integrated industry-specific business applications designed on a pure internet architecture (PeopleSoft 2004). This system was to act as the core (including management of supply chain and finance) and was complemented with different best-of-breed software modules for other more specific functions.
According to the CIO, all new IT solutions were deployed and in operation throughout the entire organization on time and on budget (within the allowance variation of +/-5%). All the defined ERP requirements were achieved to an appropriate extent. The CIO explained that due to changed assumptions and indicators, the requirements changed throughout the implementation phase. However, even if these changes were not explicitly expected, the Shipping Company did expect changes of some sort to happen throughout the project duration as "the world is not static" (CIO 2004). Some of the effects that they had not explicitly accounted for were; the improved visibility, i.e. the transparency of all business processes which imply that "if you don’t do your job, this becomes clear straight away” (CIO 2004). The project also had the effect that it challenged middle management to become more engaged in the technology than initially expected. Challenges brought forward by the ERP deployment were mostly in terms of the actual organizational changes, employee attitudes and middle-management reluctance and inability to deal with the new technology due to insufficient competence. The challenge of low IT maturity among top management also emerged in the deployment process. Other negative effects included errors in the standard package from PeopleSoft, slow deployment of the system by employees and lack of understanding for the need for change.
According to the CIO, the Shipping Company is satisfied with the ERP system and he argues that the ERP venture met their expectations and in some cases exceeded the expectations and positively surprised them. Although some negative ERP effects occurred which were not expected, the CIO said that they were aware that unexpected events could and would take place.
Hence, a conclusion drawn from the Shipping Company’s experiences and the CIO’s interview is that there is support for the hypothesis that more realistic expectations lead to a higher level of satisfaction. Even if the expectations are not explicitly and exactly identified, the awareness that unexpected events will occur, contributes to a more realistic understanding of the effects an ERP system will have.
The dimensions of Strategic and IT Infrastructure were not explicitly stated in the interview with the CIO. However, these dimensions were included in the questionnaire. The results show that even if the CIO considered some of the associated effects to exceed expectations, on an aggregate level, these effects did not influence the overall satisfaction with the ERP deployment to the same degree as the Managerial, Operational and IT project dimensions. This result supports the hypothesis (H1) since the CIO had realistic expectations of the Managerial, Operational and IT project dimensions, and it was these dimensions that contributed the most to the overall satisfaction with the ERP deployment.
The need for an ERP solution was something that had internally grown
within the Oil Company as the organization developed. According to the
RM there was no real alternative than to employ an ERP system in order
to efficiently and effectively deal with the Oil Company’s operations.
The RM argues that the main driving force for IT solution deployment is
the same today as 50 years ago – “how to get a product through the
value-chain as efficiently as possible”
(RM 2004: Interview). In 1994 the Oil Company started to implement
integrated systems to handle operations as their previous strategy
where everything had been developed in-house was no longer considered
feasible. This integrated approach was based on a notion that “a
fragmented system leads to a fragmented process”
(RM 2004: Interview), and hence a system enabling integration became
vital. The Oil Company’s explicit expectations are categorized
according to the following dimensions;
Strategic - To provide a foundation to change in order for the Oil Company to become more efficient and effective.
The oil company choose SAP as the core system, and just like the shipping company’s ERP solution, the oil company’s ERP system was complimented with different modules for other, more specific functions based on best-in-breed solutions.
In contrast to the Shipping Company, the Oil Company had a more “muddling-through” approach (Lindblom 1995) to their ERP deployment, where incremental changes were made rather than a complete overhaul of the status quo. This was both in terms of the implementation process and the ERP deployment objectives, i.e. their expectations. This might be explained by the fact that SAP (and ERP systems in general) was much less flexible in 1994, when the Oil Company started implementing it, than in 2001, when the Shipping Company deployed their ERP system (RM 2004). This implies that comparing the actual outcome in terms of ERP effects in relation to pre-deployment expectations may be difficult as expectations were formed throughout the process. Nevertheless, the main overall objectives were all realized outcomes of the ERP deployment, i.e system integration, provision of a long-term platform and support for organizational change resulting in increased efficiency and effectiveness. An effect that was unexpected but contributed to increased satisfaction was the structured and automatic way upgrades were handled within the system.
In general the RM argued that the Oil Company is fairly satisfied with the ERP solution, even though some persons in the organization would like more flexibility and functionality. However, the RM also stated that the Oil Company had not planned the ERP deployment well enough and that this contributed to unrealistic expectations.
Results from the questionnaire reveal that Operational effects contributed the most to overall satisfaction with the ERP initiative. This maintains RM’s notion of value-chain efficiency as being the main driving force for the Oil Company’s ERP investment. While the CIO in the Shipping Company was either satisfied or delighted with all met effects, this was not the case with the RM. The RM responded that the effects of Removing the need for end-user tools and Increased Cross Functional Understanding did not meet the Oil Company’s expectations, thus making them dissatisfied with these effects. However, both these effects scored low on the impact they had on overall project satisfaction. Based on the interview and the questionnaire results, the Oil Company is less satisfied with the ERP initiative than the Shipping Company. Also, in comparison to the Shipping Company, the Oil Company had less realistic expectations of the ERP effects. Therefore, the hypothesis (H1) is supported, i.e. a company with less realistic expectations is less satisfied.
The analysis of pre-deployment expectations and post-implementation satisfaction presented in the previous section will in this part of the chapter lay as a base for the refinement of research model.
In general, the research model proposed in this chapter was found to have relevance, and the hypothesis (H1) was supported. Although some strategic effects in the research model were considered non applicable by the RM, they were considered applicable by the CIO. Therefore, there the authors see no reason to remove these effects from the proposed framework.
An expected effect that was not included in the research model, but was included in the CIO’s requirement review, was that the new IT solution should improve support to IT-users. This effect may be attributed the Organizational dimension. As mentioned earlier in this chapter, the Organizational dimension of ERP effects is concerned with organizational learning and execution of strategies (Shang and Seddon 2000), and support to IT-users is argued to apply to users across the entire organization, placing it in the Organizational dimension.
Another effect that was not included in the framework, but does affect the satisfaction, is expecting the unexpected. According to the CIO (2004), because the Shipping Company was aware that unexpected events could and would occur, their overall satisfaction was not as affected by these unexpected events as much as it would be under other circumstances. As unexpected events may occur in any of the dimensions of the research models, this variable may be attributed to all dimensions, both in terms of ERP effects and in terms of customer satisfaction. However, the nature of such unexpected effects must surface before they can be placed in any of the proposed dimensions in the ERP Effects framework (Appendix 1: Table 1).
Reduced employee turnover was also expected by the Shipping Company and not included in the ERP Effects framework (Appendix 1: Table 1). The authors of this chapter argue that this effect is associated with effect of resource management (point 2.a in Table 1). This argument is supported by Barney’s (1991) resource-based view of the firm, which considers employees as resources.
A further effect identified by the CIO was improved visibility of employee performance. This effect is attributed to the Managerial dimension in the ERP Effects framework (Appendix 1: Table 1) as the Managerial dimension encompasses control of resources, i.e. employees, and monitoring of operations (Shang and Seddon 2000).
The RM identified an additional effect to the IT Infrastructure dimension in terms of a structured and automatic way to handle upgrades, i.e. ease of upgrades.
A refined ERP Effects framework, including the effects discussed in this section, is presented in Table 2 (Appendix 5).
As the results from the study undertaken support the hypothesis that more realistic expectations lead to more satisfied customers, the authors of this chapter argue that the practical implications of the findings emphasize the need for ERP vendors to manage their customers’ expectations to become more realistic. Hence the customers would become more satisfied. As discussed in the methodology, the authors of this chapter have undertaken an interview with a CEO of a business consultancy specializing in ERP deployments and the outcomes of the interviews with the ERP customers (CIO and RM) were discussed. By combining the outcome of the interview with the CEO with relevant theory, this part of the chapter will forward practical implications for ERP vendors (which could also be relevant for ERP management consultants).
Overall, ERP vendors should use communication to shape customer expectations (Sheth and Mittal 1996). Shaping expectations is understood as changing cognitions (i.e. expectations) and not necessarily as modifying behavior, although altered behavior may be an outcome. Rather, altering expectations aims “to rectify an expectation associated with the behavior so that the experienced outcome is evaluated more positively.” (Sheth and Mittal 1996, p. 141).
Shaping customer expectations could be accomplished by:
Providing training and education sessions for prospective customers in how ERP systems work and what implications they will have for the organizational structure. (Sheth and Mittal 1996; CEO 2004). The CEO also argued that pilot sessions, in which the customers are able to try the system, enhance understanding (and hence realistic expectations). The CIO also talked about the usefulness of pilot sessions, where the Shipping Company required ERP vendors to provide small, tailor-made demonstrations of their ERP solutions.
Feedback and reward systems may also be put in place to alter expectations (Sheth and Mittal 1996). In order to make the ERP customers to understand the importance of the corporation of their own employees, the consulting firm usually allocates a part of their budget to a bonus for the ERP customer’s employees if the ERP project progresses according to plan (CEO 2004).
Frame the ERP system as a way of doing business (Davenport 1998), rather than framing it as a software package (Davenport 1998). Here the ERP vendor may bring change management and business process reengineering consultants when meeting a prospective customer. This is in order for the customer to indirectly understand and expect something else than “only” the technological aspect implementing ERP software to take place.
Emphasize that the customer must comply with the requirements of behavioral prescriptions (Sheth and Mittal 1996), e.g. the ERP vendor could require that they have to be in charge over certain business aspects of the ERP implementation process in order to ensure progress and limit scope creep. For the customer, this implies that progress might be troublesome and scope creep likely to occur (CEO 2004). Other behavioral prescriptions which may be used by ERP vendors to make customer expectations more realistic are group and societal norms. Here the ERP sales person may use group pressure to induce the expectation that specific activities are to be performed (Sheth and Mittal 1996). In relation to this, the CEO (2004) argued that prospective customers, through reference meetings where they visit other companies which have deployed similar ERP systems, achieve a greater understanding of ERP effects based on the experiences of others.
Even if all the above examples may be used to alter expectations, Miller (2000) warns for the possibility to create too low expectations. This might be counter productive since low expectations may lead to a self-fulfilling prophecy of low performance evaluations. This is mentioned earlier in this chapter, i.e. that unrealistically low expectations lead to lower levels of satisfaction (Szajna and Scamell 1993). For this reason Miller argues that “IS professionals should create high expectations that are achievable, realistic, and unambiguously communicated” (Miller 2000, p. 93). In order to do this the ERP professional must first identify what customers expect via customer contacts, formal surveys and analyzing the industry in which the customer is operating (Miller 2000). The latter is considered as an area that the Oil Company’s ERP providers were lacking (RM 2004), and something that CEO (2004) emphasized as important when discussing ERP solutions with customers. Secondly, the expectations should be evaluated in order to find out what expectations are feasible to alter and what the ERP vendor realistically can deliver (Miller 2000; Sheth and Mittal 1996).
In relation to the importance of realistic expectations, the CEO (2004), who has extensive experience in ERP implementation projects, argued that realistic expectations are important only to a certain extent. He argued that “if the customer would really know what they were getting into [ERP project] – they would never do it!” (CEO 2004). Yet, when CIO and RM were asked if they would have implemented the ERP solutions even if they would have known then what they know now, both answered that they would have gone ahead with their ERP deployment, and both CIO and RM considered realistic expectations to be important for satisfaction.
Even though all effects in the ERP Effects framework (Appendix 1: Table 1) were supported and found relevant for the current research, this does not necessarily mean that these effects are generalizable. However, the research undertaken in this study is of a qualitative exploratory nature, aiming to provide insight, ideas and understanding about specific phenomena. Future research encompassing a larger sample could be done in order to generalize findings, to disconfirm effects, or to find support for additional effects or dimensions.
Limitations to the research undertaken in this study will be elaborated on in next section, however, one of the main weaknesses lies in the fact that the interviewees might not have in-depth insight and understanding of the different areas of their organization. They may therefore be unable to adequately evaluate all the ERP effects in the organization. This weakness has implications for future research, and further studies are recommended to limit the dimensions and effects discussed in the interview to areas of the business that the interviewee is directly involved with and deeply knowledgeable about. Another suggestion is to keep the research model with all the proposed dimensions and effects, but to interview more people in the organization regarding the areas they are knowledgeable about.
Another suggestion for future research is spurred by the practical implications of this study. Elaborating on the proposed strategies (see Practical Implications) and outlining the types of actions an ERP vendor may undertake in order to manage customer expectations, as well as their relative effectiveness, are forwarded by the authors of this chapter as a relevant area of study for further insights into customer satisfaction with ERP systems.
The authors of this chapter consider the main weakness of the study to lie in that the interviewees have been asked to answer and discuss the ERP effects, expectations and satisfaction on behalf of their organization. The weakness lies in the fact that the interviewees might not have in-depth insight and understanding of the different areas of their organization in order to be able to evaluate all the ERP effects in the organization adequately. Nonetheless, due to the scope limitations of this project the interviewees chosen were considered the most appropriate owing to their overall knowledge about IT solutions and ERP effects and hence deemed to represent the companies in a sufficient manner.
Another weakness lies in the position the interviewees have in their organizations and that this might lead to biased outcomes of the interviews/questionnaires. This is because the authors asked the IT professionals about their expectations in comparison to actual outcome, indirectly questioning these persons’ abilities to correctly evaluate the risks and returns of the ERP venture. In other words, if there is large discrepancy between the expected and perceived ERP effects, the person responsible for the evaluation of the ERP business project (the interviewees) may be accused of not evaluating the project adequately. Subsequently, this might lead the interviewee to shape his response in a manner that does not undermine his position. Although this is a weakness of this study, these persons were considered to be appropriate interview subjects as they were expected to be the most knowledgeable persons regarding the companies’ IT solutions. Moreover, in order to limit this weakness, the authors of this study, prior to the interviews, told the subjects that the source of discrepancies was likely due to insufficient expectations management on behalf of the ERP vendors and unpredictable factors.
Possible weaknesses might also surface on behalf of the issued questionnaire. One evident claim might be that many of the effects proposed in this chapter, although well argued for in existing research, may not have been well understood by the interviewees. The authors of this chapter believe that it is likely that if a respondent did not understand the question/effect, he would check ‘Not Applicable’. However, ‘Not Applicable’ would also be the best response if there was no motivation on behalf of the organization to consider the proposed effect. Nevertheless, the CIO responded on all effects, while RM chose ‘Not Applicable’ on five effects.
In the marketing area of study, scholars have long argued that companies should meet customer expectations in order to attain customer satisfaction as satisfied customers are more valuable to the firm. In the IS arena, where failure rates are alarmingly high, companies struggle to meet expectations posed on the systems. However, ERP customer expectations have been argued to be too high in terms of system capabilities and ease of implementation. In this case ERP vendors need to shape their customers’ expectations to become more realistic rather than over promising ERP performance, since the latter results in low customer satisfaction. Some ERP vendors have circumvented this problem by deliberately setting expectations too low. However, there is an evident trade-off between setting expectations too low (in order to create customer satisfaction) and attracting customers. Therefore, shaping customers’ expectations to reach a realistic level would be in the ERP vendors’ best interest. The fairly unconventional notion that unrealistically low expectations correspond with lower levels of satisfaction further underpins the argument that realistic expectations lead to raised customer satisfaction and also strengthens the argument that ERP vendors/consultants should manage their customer expectations to become more realistic.
The result of the study undertaken by the authors of this chapter supports the hypothesis that more realistic expectations lead to increased customer satisfaction, both when expectations are unrealistically high and unrealistically low. Hence, ERP vendors and business consultants specializing in ERP are advised to manage their customers’ expectations in order to make them more realistic.
Based on a mix of primary and secondary sources, including both scholars and practitioners, some strategies for changing customer expectations to become more realistic, thus increasing customer satisfaction, have been forwarded. However, the authors of this chapter call for future research concerning effective expectation management in an ERP consumer context.
To conclude the authors maintain that unrealistically low expectations correspond with lower levels of satisfaction because low expectations may become a self-fulfilling prophecy of low performance evaluations. Furthermore, although unrealistically high expectations might lead to lower satisfaction, high expectations do not necessarily guarantee dissatisfaction. Therefore, the authors of this chapter agree with Miller’s argument that “IS professionals should create high expectations that are achievable, realistic, and unambiguously communicated” (Miller 2000, p. 93).
Appendix 1 : Table 1:
The ERP Effects framework
Appendix 2 : Questionnaire for the
CIO and the RM
Appendix 3 : Interview guide
for the CEO
Appendix 4 : Interview guide
for the CIO and the RM
Appendix 5 : Table 2: The ERP Effects
framework (refined)
Information technology has in a short time become a success factor within most organizations. Earlier the IT-department often was a sub department of the financial department, with anchoring in cost efficiency, but in the last few years’ information technology has gotten more attention and many companies have restructured their organizations. A consequence of the arrival of IT, increasingly more companies have to integrate IT and business processes in their organizations.
In the increasingly competing business environment the need for effectiveness and efficiency in organisational performance is more apparent then ever. How can organizations increase the organizational performance due to the effects that they plan to get when implementing an Enterprise Resource Planning (ERP) system? A lot of implementation of IS systems fail or the business don’t get the benefits that they had hoped they could get after implementation (Lauden & Lauden (2000). Why don’t the organizations get the benefits out of a system? It is known that if implementing an ERP system the business has to change to fit the system or change the system to fit the business. This could be seen on as the key problem when implementing a new system, that’s why the effects that could give benefits to an organization must be planned well before implementing. It isn’t easy to predict all effects of a system and some unplanned effects can occur after implementation. This paper will look upon the expected effects before and the perceived effects after implementing, both positive and negative.
There has been a lot of research on this topic, something that shows that this specific area within the MIS field is of special importance. Kohli and Devaraj have found a lot of IT payoff effects and benefits that is driven by IS systems, some effects could be: cost reduction, inventory reduction, labour productivity.
Vendors of ERP systems (SAP 30%, Oracle 14%, Peoplesoft 7%, market shares worldwide in 2000) argue that the solution for meeting the increasing demands from customers, legislations and partners is to implement ERP systems (Gilbert, 2000, Shanks, G. and Seddon, P. 2000). In this paper we will discuss some issues regarding implementation of ERP systems, their characteristics, probable effects and constraints. The paper is divided into 3 sections. The first section is a theoretical discussion of implementation of ERP and organisational performance. We use the division of organisational performance efficiency and effectiveness according to Melville et al (2004). The division of organizational performance is neatly described in the theoretical discussion of the paper. Secondly we will put forward a research model, a research question and some propositions. Finally we will develop an interview guide which will be used on two managers, which have implemented and are currently using, an ERP system in Norway.
In this paper we will investigate if there is any
correlation of expected effects and perceived effects at an
organisational level with use of ERP systems. This leads us to the
following research question.
Constructs in this research question are use of ERP systems and operational effects. Variables in use of ERP are integration and automation of business processes. Subsequent, we want to investigate the correlation between planned an unplanned effects in organisational performance as an addition to only find the gap between expected and perceived effects. This leads us to three propositions.
The major vendors of ERP systems have build and bundled new functionality into their packages and increased the robustness of their systems. The span of automation and integration makes the interaction between the different business processes seamless. This again leads to a more efficient business. This leads us to the first proposition. Proposition 1: The use of ERP- with integration and automation of business processes, increases organisational performance at an operational level for a firm.
Because of the ERP vendors increasing experience and increasingly enduring packages we believe that implementing ERP systems will increase the operational performance. This drives and matures ‘best practises’ in different industries and business processes. This leads us to the second proposition. Proposition 2: The use of ERP¬¬¬- with the use of ‘best practises’ increases the perceived operational performance gain.
Organisational performance can be divided into two formulations, efficiency and effectiveness (Melville et .al 2004) We believe that planned effects are more correlated with efficiency, since these effects are the incentives for the use of an ERP system. We also believe that unplanned effects are more apparent with effectiveness, since these effects are related with the business external environment. This leads us to the third proposition. Proposition 3: The efficiency of an implementation of ERP system is correlated with planned effects and the effectiveness of an implementation of ERP system is correlated with unplanned effects.
Positive findings in this research may provide businesses a more shaded picture of the benefits of implementing and use an ERP system. For further research this study may bring new practical light on how the business perceives the outcome of use an ERP system.
Positive findings can also contribute to better knowledge of which effects an IT system can have on an organization. With use of expected effects up against the perceived effects, we could find out which of the expected effects who has been perceived and those not. In the outcome here could also be some unplanned effects that weren’t expected at all. The integration and automation of business processes, increases organisational performance hence to better operational effects throughout the whole organization. Some positive findings of automation could be cost reduction, inventory reduction and operating costs. Earlier in this paper it was stated that use of “best practice” systems or modules could increase operational performance when implemented. Positive findings here would be if all the expected effects is perceived after implementation, but there could also be negative findings, and these could be that few or none of the expected effects are perceived.
Often the ERP systems can not give businesses all the operational benefits and effects that the vendor promises and the businesses expect. What have gone wrong in the process when effects fail to appear? Maybe the organization did not plan the implementation well enough. Negative findings could be when effects fail to appear, but could also enlighten what went wrong in the process to use in other projects and implementations.
This section will discuss operational performance with use of IT, some aspects of implementation of ERP systems and finally interlink these two. The theory is derived from writings about operational performance with use of IT and theory from implementing and use of ERP systems.
Previous research has shown that information technology may contribute to the improvement of organisational performance (Brynjolfsson and Hitt 1996). Use of IT could impact on productivity enhancement, profitability, cost reduction, competitive advantage, inventory reduction (Devaraj and Kohli 2003). Other measurements for organisational performance could be increased market share, customer satisfaction and supplier satisfaction as the competitive performance improves due to increased internal efficiency (ibid).
There is a lot of focus and initiative in organizations today to find better ways to optimize operational performance with use of IT. This includes providing relevant and timely information so organizations can improve the efficiency and quality of their business operations. Operational performance is all about the practice of understanding, optimizing and aligning the operational business activities and processes to a common set of goals and objectives to improve effectiveness (Smith 2003).
In this paper the focused area will be on use of two formulations on organisational performance. First is the efficiency, which emphasizes on internal metrics such as cost reduction and productivity enhancement. Second is the effectiveness, which focuses on the firms’ external environment (Melville et al 2004). Based on a literature review encompassing the above mentioned authors we have found these measurements for efficiency and effectiveness and is described in the table below.
Table 1 Measurements for organisational performance
ERP systems integrate organizational processes through shared information and data flows (Shanks and Seddon 2000). ERP integrates and automates some/all of the business processes (i.e. manufacturing, logistics, distribution, inventory, invoice and accounting).
Characteristics of ERP systems are that it is packaged software consisting of modules for specific business purpose. There exists different modules for different purpose and a lot of these modules can be based on ‘best practises’ from leading industries, for instant SAP. When using cross sectional modules (i.e. finance and logistics) it will streamline your business processes. However, these modules put constraints on additional wanted functionality. ERP integrates business processes such as manufacturing, logistics, distribution, inventory, invoice and accounting.
Some argued effects of an ERP system is that it will integrate information and processes, standardize and speed up manufacturing processes and reduce inventory. However the implementation can be problematic, which cost lot of money and consumes time and resources (Shanks G. and Seddon, P. 2000).
ERP systems are mainly suited for larger companies since the initial cost are high and therefore need for a large volume to cover this cost (Davenport 1998). But there are smaller vendors that sell smaller modules to small and mid sized companies.
The effects of ERP implementations it is much similar to implementation of large IT systems considering organisational performance (Davenport 1998). However, you want an ERP system for your company, but it seems you have to make some specific changes to it, which is special to your company. Then you must consider making the system fit the company or the company fit the system. If you change the system there are some risks. As mentioned above it could be difficult or more expensive to install later upgrades. However, to get a focus on the business process, instead of the system, and improve the ways of doing business (Davenport 1998). These considerations must be taken into account if wanted the best possible effect of an IT investment.
An outcome of any IT implementation can be placed into four categories (see matrix below). This could be both planned and unplanned positive- and negative effects. The planned effects are to a large extent the incentives for the investment. However, planned- and unplanned effects are both contributors to the realized value of an IT implementation.
Table 2 the effects matrix, Gottschalk 2000
This matrix shows examples of what we could possible find when conducting our short survey.
The constructs in this research are operational performance with use of IT, some aspects of implementation of ERP systems and finally interlink these two.
Based on theory discussed, the research question and the propositions, a detailed research model can be drawn. The model shows how implementation of ERP systems may influence organizational performance, and include the constructs expected effects and perceived effects. We can see the constructs expected effects and perceived effects. The detailed research model is described below.
Figure 1 The Detailed Research Model
In this research we will use a qualitative approach. Our unit of analysis is companies that have implemented and uses ERP systems within their organization. The research will try to find the expected and the perceived operational effects before implementation and when using an ERP system. The accuracy of the operational effects is dependent upon the quality of data utilized, hence the data source is critical to efficiency and effectiveness analysis, and so there is a need to gather first hand data, not secondary data from recent studies. Our study can have varying levels of data aggregation, like time and which product implemented. The time of implementation and the vendor who delivers the product can have something to say for our analysis. If the implementation has been done recently the benefits and disadvantages could not have been spotted yet. Therefore the population for our survey is persons in an organization who have done one or more implementations of ERP systems.
We want to interview two companies (one person in each company) that have been through an implementation of an ERP system and still use it. We believe a qualitative approach is best suited for this analysis. This is because the responses from the respondents could be in a diverse fashion and it is up to us to draw the main line and conclude on their responses.
We have taken the basis from other IS surveys and looked on how they have formed their questions to help us make ours. We have used open end questions, which are most suitable for this type of survey.
The participating organisations in this research are the grocery retailer Coop Norge and the car retailer Møller Group.
Coop Norge got a market share of approximately 23-25% in the Norwegian grocery retailing industry. As a retailer they distribute groceries from different producers to the stores all over the country. We interviewed their CIO Per Vestby. Coop Norge implemented SAP during 1997-98 and still uses it. They implemented the Retail version and the CIO of Coop claims that it was the largest implementation of Retail version in the world at that time. The main purpose of their implementation was the need and wish for better support of their entire value chain.
The Møller Group is one of Norway’s largest automotive companies. They are engaged in activities such as import, retail sales, service and financing of Volkswagen, Audi and Skoda. We interviewed their CIO. The Møller Group has developed their systems for their core businesses in-house because there is no business standard or no standardized packages to buy from the different vendors.
When interviewing the two CIO’s we used a questionnaire as a basis (see appendix 1). However, as the interview progressed it was not always applicable to ask all the questions we had prepared. As a result of this we let the respondents talk along and we provided them with some follow up questions if needed. The results of the two interviews can be summarized in the table below.
Table 3 Outline of the results from the interview
In terms of incentives for implementation of ERP systems we can read from the results that the two respondents experience some similarities. Both respondents claim that the use of an ERP system increases the organizational performance at an operational level for a firm. This is in line with the first proposition.
Initially the system that Coop needed was not available and they needed to do some adjustments. They did to some extend adjust the original software to fit their needs. However, the CIO of Coop now believes that the Retail solution from SAP is better then the one they got now, offering more functionality and is cheaper to use because of the low need for adjustments. In the case of The Møller Group they stated that in the near future they will take a closer look on what SAP has to offer since they have developed a solution special made for the car retailing business. These two findings can somehow imply that the business perceives and approves the ‘best practises’ delivered from the vendors. In terms of the maturing of ‘best practices’ from the ERP vendors and what the two respondents states, we can somehow conclude that the second proposition is supported.
When we read our results it is difficult to make sense of our third proposition. The cause of this could be the difficulty (at least to a greater degree then we believed) of defining the responsibility for the different effects. The further discussion in this paper will leave out the third proposition.
The table below summarizes the proposition results.
Table 4 Proposition support
Recalling the research question: What is the correlation between expected effects and perceived effects with use of an ERP system? In the case of Coop they have a high correlation between the expected and the perceived outcome of their use of the ERP system. In the case of The Møller Group it is also correlation but not stated in a clear manner. However, two of our propositions is supported in both cases.
An implication for practice is that our study may contribute to enlighten decision makers to make better decisions when deciding upon a ERP implementation.
Our major findings are that we have founded some support for two of our propositions. When implementing and use of ERP system the traditional effects (i.e. reduction of cost, reduction of inventory etc) is supported. There are to a small degree other implications of the use of such a system. Another finding in this research is that businesses endorse the maturing solutions from ERP vendors. They believe that buying standard packages with a minimum of adjustments is better and more robust then doing in house-development. Presumed that the standard package delivers the functionality wanted.
Limitations to this work is that it is only conducted two interviews. However, this research would be strengthened with a broader span of respondents. A second limitation is that the respondents could be biased to be overly positive with the perceived effects of ERP system use.
We think this paper have been an excellent paper to work with and write this autumn, because we have gotten relevant interview training and had the chance to do this before we start on our thesis work. It have also been a good thing to present the ongoing work in class, because we got a lot of good feedback and that helped us to improve our paper. We will also send some greetings to Frida, Ottar and Oleg for commenting our paper so we could make it better. But we had some problems with the interview part of the paper. It was very hard to find respondents to interview or had the time and opportunity on so short notice, due to our short deadline on the paper. We sent out a lot of e-mails, but we only got one answer, it was from Møller. But we called Coop and the IT director of Coop agreed to meet us for an interview. To get interviews took a few weeks, so there has been short time to analyse the data we got. We found out that we had to find respondent actively instead of passively with mail. We had some problems with the interviews and the problem that occurred was that we operated with two terms of operational performance, effectiveness and efficiency, and it was hard to get the respondents to understand the differences between this to terms. We should have made some of the questions more clear and precise.
To sum up, we think the project has gone well, but like mentioned we had some problems with the interview part of it.
BACKGROUND INFORMATION 1. Background information on the respondent and the organisation
a. Name, company and position? b. Did you buy a packaged solution or developed in-house. i. Which vendor has delivered your ERP system? ii. Which packages did you buy? c. Did you develop the system in-house? i. To which extend did you use consultants when implementing the system? d. What was the scope of the implementation? Which processes did you integrate with your ERP system? How long did the implementation take? e. How would you describe the level of difficulty of the project? ORGANISATIONAL EFFECTS 2. What were the incentives for your implementation? a. Productivity enhancement? b. Cost reduction? c. Inventory reduction? d. Cycle time reduction e. Labour productivity? f. Decrease in operating expenses? g. Standardization of Business processes h. Increase in profitability? 3. Which operational effects did you experienced after the implementation of the ERP system? a. To which degree did you experience productivity enhancement? b. To which degree did you experience cost reduction? c. To which degree did you experience inventory reduction? d. To which degree did you experience labour productivity? e. To which degree did you experience cycle time reduction f. To which degree did you experience increase/decrease in operating expenses? g. To which degree did you experience increase/decrease in profitability? 4. Did the project meet the goals? a. Budget, time, functionality… STRATEGIC CHANGES 5. Did you experience an increase external linkage (supplier/customer) satisfaction? a. What was the supplier satisfied with? b. What was the customer satisfied with? 6. Did you experience a gain in competitive advantage? 7. Does your system support business growth? UNPLANNED EFFECTS 8. Did you experience any unplanned effects? a. Reduced flexibility? b. Increased costs?Information Technology is one of the most important reasons for organizational change taking place in today’s society. The effects of innovation in the area of Information Systems in general are many, and they may be planned and unplanned, as well as positive and negative. This chapter focuses on negative psychological effects that might occur when implementing and using a Knowledge Management System. This is an important aspect to consider in preventing future failure within the field of Information Technology. The purpose of this chapter is to investigate if increased use of Knowledge Databases leads to an increase in unwanted, job-related, psychological effects. The results from the research show that increased use of Knowledge Databases leads to less creativity and sharing of tacit knowledge. Little has been found concerning the relationship between use of a Knowledge Database and distress and information anxiety. The hypothesis concerning these issues were neither accepted nor rejected, but the results show indications of the phenomena needing further investigation. Through this research it has been confirmed that increased use of Knowledge Databases leads to an increase in unwanted, job-related, psychological effects.
Information Technology (IT) is one of the most
important reasons for the organizational change that is taking place in
today’s society. The development of new Information Systems (IS) to
handle a variety of job processes, in a new way, is creating
possibilities, but is also making the organization change. (Larsen
1999; Greenberg & Baron 2003; McNish 2002; Schwarz & Brock
1998) This has important implications for the organization and the
people in it. As more work is shifted to digital brains, some work that
was once performed by human brains becomes obsolete; new opportunities
arise and people’s expectations concerning various aspects of work
might change (Greenberg & Baron 2003).
The effects of innovation in the area of IS in general are many, and
they may be both planned and unplanned (Larsen 1999). This also applies
to the field of Knowledge Management Systems (KMS) (Gottschalk 2002a).
The majority of literature enhances the positive effects of KMS-use
(Business Source Premier 2004). Some literature, on the other hand,
indicates that the issue of non-beneficial effects of such innovations
is particularly important to address. IS-based research on KMS will
then develop a stronger theoretical base that includes both favourable
and unfavourable consequences of knowledge and its management.
(Schultze & Leidner 2002)
This chapter focuses on the negative effects that might occur when
implementing and using a KMS - an important aspect to consider in
preventing future failure within the field. The chapter also enhances
the
importance of linking the field of organizational psychology to the
implementation and use of a KMS when analysing possible effects. In
relation to this, the areas linked to KMS are distress, information
anxiety, creativity and the sharing of tacit knowledge, and whether
there is an increase/decrease as a result of using Knowledge Databases.
The chapter is divided into eight main sections. These are: ‘Research
Question’, ‘Theory’, ‘Framework for Analysis’, ‘Research Design &
Method’, ‘Analysis’, ‘Discussion’, ‘Limitations’, and ‘Conclusion’.
This section will present the research question and
its reasoning. The research question is based on a literature review
done in Business Source Premier (2004) showing that little has been
done in order to categorize the negative effects and/or side effects
that the use of an IS can create. Based on an article by Schultze &
Leidner (2002), which points out the lack of looking at Knowledge
Management (KM) from both a positive and a negative side, a KMS is
chosen. The effects the system might have on the user, in this case a
Knowledge Worker (KW), will be the focus.
The constraints are a result of the necessity to, in a relatively
concrete way, describe the features of the system, and further what
kind of person a KW is – what are the typical tasks this type of person
has to deal with in the organizational environment? A more detailed
discussion of the reasoning behind will be presented in the section
‘Framework for Analysis’.
The field of IS research is wide and comprehensive. The following theory concepts are used as guidance in clarifying the theory areas relevant to the research question and research model. This section will address three main areas which are ‘IS Effects & Negative Aspects’, ‘Concepts in KM’, and ‘Psychological Factors & the Human Side of Work’.
The effects of innovation in the area of IS are
many, and mapping all effects of IS investments is a complex task.
The effects that surface during the implementation of an IS are both
planned and unplanned, and affect the organizational environment such
as social and professional relationships, enforcement of new structures
and work procedure etc. It is therefore important to include a broad
spectrum of issues in the planning of IS to ensure the success of
implementation and usage. (Larsen 1999) The majority of literature
focuses on the beneficial effects IS provides for its users, due to the
facts that IS projects tend to fail. (Sauer 1993; Laudon & Laudon
2000) To be better equipped for preventing future failures, one should
focus on the negative aspects of the IS process, to map general
unplanned effects and to be capable of reducing the failure rate.
Gottschalk (2002a) has stated that in addition to the planned and
unplanned positive effects, it is also important to analyze the
negative results as planned and unplanned negative effects. The
negative effects are important to identify because they can stimulate
ideas and actions that further can lead to more positive and less
negative effects.
The field of KM has become a popular topic for research, because of the
increased use of IS in knowledge intensive firms. Research states that
IT/IS can lead to a greater breadth and depth of knowledge processes in
organizations. The main focus is on the importance of KM processes, and
how various IT tools and capabilities can play a variety of roles in
supporting these processes. (Alavi & Leidner 2001) The use of an IS
for knowledge creation and sharing is seen as being highly beneficial
to the employees and to the organization as a whole. In IS literature,
most research on KM assumes that knowledge has positive implications
for organizations. However, knowledge can be seen as a double-edged
sword (Schultze & Leidner 2002) and thereby also KMS. Resent
research highlight the lack
of attention paid to the unintended consequences of managing
organizational knowledge. Focusing on this will broaden the scope of
IS-based KM research. (Alavi & Leidner 2001)
A research conducted by Schultze & Leidner (2002) on categorizing
research articles into different discourses, showed that the majority
of the articles were located in the normative and interpretive
discourse. This implies that there is a tendency of adopting an
optimistic view of the role of knowledge in organizations and the role
of IS in enabling KM. Thus there is a need for IS-researchers to
wrestle with the difficult issues that KM might incite, and develop a
stronger theoretical base that includes both favourable and
unfavourable consequences. The debate should be on whether all effects
of an IS implementation turn out to be beneficial. Do investments in
IT/IS inhibit or facilitate knowledge creation and use? Does an
increase in the breadth and depth of knowledge result in greater use of
a KMS and greater use of available knowledge? Or contrarily, does such
an expanded availability discourage usage as the potential search and
absorption time for needed knowledge might increase? (Alavi &
Leidner 2001)
Resources that are valuable, unique and difficult to imitate can provide the basis for a firm’s competitive advantage (Barney 1991; Barney 2002). Knowledge is a resource that is becoming increasingly important in today’s business world, because the knowledge asset may produce long-term sustainable competitive advantage for an organization. Knowledge can be defined as information combined with experience, context, interpretation and reflection (Gottschalk 2002b; Davenport & Prusak 2000; Alavi & Leidner 2001; Fahey & Prusak 1998). The long-term sustainable competitive advantage comes from the company’s ability to effectively apply the existing knowledge to create new knowledge (Alavi & Leidner 2001). This is also supported in resource-based theory, which indicates that resources such as knowledge, learning and human capital are likely to be sources of sustainable competitive advantage for a firm (Barney 2001). Hitt et al. (2001) also claims that such intangible resources are more likely to produce a competitive advantage because they are often rare and socially complex, thereby making them difficult to imitate. The knowledge resource can be used several times, and it accumulates within an organization through use and combination with employees’ experience (Gottschalk 2002b). This section will briefly discuss and give definitions on KM, KMS, Knowledge Database and KW.
KM is becoming a critical business discipline as
enterprises seek to enhance competitiveness in real time and ensure the
reliability of information to support real-time decision making
(Caldwell et al. 2003). KM can be defined as “the process of gathering,
organizing and sharing a company’s information and knowledge assets”
(Greenberg & Baron 2003). It is a discipline that promotes an
integrated approach to the creation, capture, organization, access and
the use of a company’s knowledge and information assets (Gartner 2004).
It also refers to identifying and leveraging the collective knowledge
within an organization to help it compete (Von Krogh 1998). KM is
supposed to increase the innovativeness and responsiveness within the
organization (Hackbarth 1998).
A KMS refers to a class of IS that is applied to managing
organizational knowledge. It is IT-based systems developed to support
and enhance the organizational processes of knowledge creation,
storage/retrieval, transfer and application (Alavi & Leidner 2001).
It is important to keep in mind that IT is only one important aspect of
a KMS, and it also relies on human skills for success. Computers merely
organize what the skills are and where in the company they may be
found. Simply having a KMS does not alone ensure success.
An important success-factor is getting all the employees to use it, but
very often this is not the case. For the KMS to be effective, people in
the company must both donate information to the system and receive
information from it. The tendency for employees to refrain from using
the knowledge that is available to them results in poor performance,
called the ‘knowing-doing gap’. There are several reasons for not using
a KMS; the most dominant is the tendency for employees to be afraid of
expressing their ideas or of seeking ideas from others. (Greenberg
& Baron 2003)
A typical example of a KMS is a Knowledge Database. A database might be
seen as an organized body of related information. It is a shared
collection of logically related data, designed to meet the information
needs of multiple users in an organization. Knowledge Databases can be
supported by IT; by electronically storing the information. Empirical
studies show that both organizations and individuals forget. The human
memory has limitations, while computers never forget. (Gottschalk
2002b) A Knowledge Database is an electronic storage of knowledge
within an organization, which is helping the company to organize and
share the knowledge between the employees. A Knowledge Database can
also be known as knowledge storage (Gottschalk 2002b).
A KW can be defined as “an employee who is able to
find, understand and use knowledge in the organization on his or her
own” (Gottschalk 2002b). KWs are people whom companies trust to make
decisions within their respective domains (Gartner 2004). They are
changed by the information in their environment, and they in turn seek
to make changes through information. Diversity and ad hoc behaviour
patterns are common in knowledge work. Examples of KWs are lawyers,
marketing analysts, engineers, product developers, resource planners,
researchers, and legal counsellors.
In relation to KWs, a common distinction is made between explicit and
tacit knowledge. Explicit knowledge can easily be codified and is
possible to express in words and numbers. Tacit knowledge, on the other
hand, is highly personal and hard to formalize. This makes it difficult
to communicate or share with others. Tacit knowledge is deeply rooted
in an individual’s actions and experience as well as in the ideas,
values and emotions. (Gottschalk 2002b; Greenberg & Baron 2003;
Nonaka 1991)
KWs are highly dependent on, and use the tacit knowledge in main parts
of their work. For KWs it is important to be creative and innovative in
their daily work, in order to provide the organization with the best
results possible. A dangerous pitfall is the fact that organizations
often pay too little attention to the role and importance of tacit
knowledge. It has been argued that tacit knowledge is more important to
an organization than explicit knowledge (Fahey & Prusak 1998).
Tacit knowledge provides a higher probability of creating strategic
value than explicit knowledge (Hitt et al. 2001; Gottschalk 2002b;
Barney 2001).
The theories and concepts discussed in this
section, is found under the umbrella “Organization Psychology &
Organizational Behaviour”, which focuses on increased knowledge about
all aspects of behaviour in organizational settings, human and society.
Psychology is the scientific study of behaviour and mental processes
and how they are affected by an organism's physical state, mental state
and external environment – in relation to the mind and the processes of
the mind. Three levels of analysis are used in organizational
behaviour; individual, group and organizational processes (Greenberg
& Baron 2003). This chapter focuses on individual processes.
KM and the use of KMSs are influencing organizational change, and
further what this implies for the employees. ‘Knowing-doing gaps’ may
occur, and this can lead to bad performance. Using such systems might
create psychological effects that are far from good. This section
presents theories and definitions about some mental processes that can
transpire through use of a Knowledge Database. These processes are
distress, information anxiety, creativity and sharing of tacit
knowledge.
Stress is by most of us associated with something
negative. But stress can be viewed as both positive and negative, and
it is important to clarify what positive stress is in order to better
understand negative stress. Positive stress has the effects of spurring
motivation and awareness, and providing the stimulation to cope with
challenging situation. It also provides the sense of urgency and
alertness needed for survival when confronting threatening situations.
Positive stress is also called eustress, while negative stress is
called distress. "Positive stress is what turns you on, while
negative
stress is what wears you out." (Johnston 2004) In relation to
positive stress, Anderson
& Arnoult (1989) found in their research that it appears to be
little evidence that positive stress affects health, while negative do.
Based on this the focus will be positioned on distress.
Job distress has been a costly and disruptive problem for
organizations for decades, and it shows no signs of diminishing any
time soon (Perrewe et al. 2004). Stress is the pattern of emotional
states and physiological reactions occurring in response to demands
from within or outside an organization (Greenberg & Baron 2003).
Distress is an all too common part of life today, and something that
few
manage to avoid. Research has shown that high level of stress adversely
affect physical health, psychological well-being, and many aspects of
task performance. In this relation it will be increasingly important to
point out factors that lead to distress in order to eliminate them.
Stimuli that create distress are known as stressors and these are
formally defined as any demands, physical or psychological, encountered
during that course of living. It is incumbent upon organizational
scientists to develop a more informed understanding of the factors that
can protect people from the negative consequences of job distress
(Perrewe et al. 2004). Occupational demands are one cause of distress
in
the workplace. Repeatedly exchanging information with others (Greenberg
& Baron 2003), as in the case with sharing knowledge in a Knowledge
Database and making it available, can be a great stress factor.
Overload is a familiar problem when it comes to the
management of information. Information is not only essential but also
widely available, and people are faced with more information than ever
before. This has created a phenomenon called ‘information anxiety’,
which is pressure to store and process a great deal of information in
our heads and to constantly keep up with gathering it (Greenberg &
Baron 2003) - “the black hole between data and knowledge”
(Wurman
1989). This can also be related to Knowledge Databases. If there is no
system of what is being collected, the database rapidly becomes huge,
and there will be difficult to get an overview and find what is
important and relevant. This can create information anxiety.
Wurman (1989) presents in his book an interesting observation regarding
information anxiety: “One of the most anxiety-inducing side effects
of
the information era is the feeling that you have to know it all.
Realizing your own limitations becomes essential to surveying an
information avalanche; you cannot or should not absorb or even pay
attention to everything.”
Creativity is defined as the process by which
individuals or teams produce novel and useful ideas. Creativity is
composed of three basic components: domain-relevant skills,
creativity-relevant skills and intrinsic task motivation (Amabile
1983). There are ways to develop a creative work environment. These
approaches may be providing autonomy, allow ideas cross-pollinate,
making jobs intrinsically interesting, set your own creative goals,
supporting creativity at high organizational levels, having fun, and
promoting diversity. (Greenberg & Baron 2003)
Solving problems creatively requires extensive and effortful cognitive
processing. This requirement is magnified further by the complex,
ambiguous situations in which most organizational problems occur.
Employees must define and construct a problem, search and retrieve
problem-relevant information, and generate and evaluate a diverse set
of alternative solutions. Creativity necessitates that all these
activities are completed effectively (Reiter-Palmon & Illies 2004).
Seeing this in relation to using a Knowledge Database, creates the
question if such a database inhibits or facilitates creativity?
New knowledge always begins with the individual.
Making knowledge available to others is a central activity in all
knowledge intensive firms. In sharing knowledge within an organization
it is necessary to convert the tacit knowledge into explicit. According
to Nonaka (1991) the process of converting tacit knowledge can be
described the following way: “converting tacit knowledge into
explicit
knowledge means finding a way to express the inexpressible”.
Because
tacit knowledge typically is shared through discussion, stories,
analogies and person-to-person interaction, it is difficult to capture
or represent in explicit form (Gartner 2004).
One important aspect to consider when investigating the negative
effects of a KMS is the degree of knowledge-sharing with respect to
tacit knowledge. Tacit knowledge can be defined as the personal
knowledge resident within the mind, behaviour and perceptions of
individuals (Gartner 2004). It is embedded in an individual’s actions
and commitment to a specific context, which makes it extremely
difficult to capture by an IT-system. Since tacit knowledge consists of
mental models, beliefs and perspectives so integrated that it is taken
for granted, it cannot easily be articulated.
When introducing a KMS, the individual sharing of knowledge changes
dramatically. The individual connection and relation between co-workers
are no longer as strong as it used to be. The social support within the
organization might decrease, leading to substantial loss of tacit
knowledge in the process. The lack of social support might result in
isolation, and the costs of isolation might inhibit information
sharing. Talking to other people can help us learn about ways of coping
with problems and give us new perspectives. (Greenberg & Baron
2003)
This research is inspired by Schultze and Leidner (2002), and is a dialogic discourse approach. It examines the contradiction in managing knowledge. The process of searching for literature and theories as foundations for the research model was complex. Research related to the field of KM is extensive, but the main focus has been optimistic. A Knowledge Database, as one type of KMS, has been chosen to limit the research area. In order to get a picture on how IS effects in general are influencing an organization and its employees, other areas of research were analysed. Similar ISs seen from different angles and same thematic were used as terms to search for effects related to the research question. There are few sources on negative effects of IS implementation in general. The effects that have been topic for research are mainly pre-usage. This means that the effects are not directly related to the psychological aspects of the usage-phase. The focus tends to be on whether the users actually use the implemented IS and what affect their intensions to use. (Fagan et al. 2003-4; Venkatesh et al. 2003) The following sections will present an explanation of constructs and variables used in the research model, and a graphical research model.
The research model is build with the construct ‘A
limited and B rich’. ‘Use of Knowledge Database’ is the independent
construct in the model and can be defined as the process of using a
Knowledge Database as a significant part of daily work routines. Within
the independent construct is the variables ‘degree of use’, ‘perceived
usefulness’ and ‘user satisfaction’. The inspiration behind is DeLone
and McLean’s (1992) “Model of IS Success” (Appendix
1) and Seddon’s (1997) “Respecified version of DeLone and McLean’s
(1992) Model of IS Success” (Appendix
2), to see how these three variables together have an individual
impact on the KW related to the organizational benefits and future
usage.
The independent construct leads to the dependent construct, which is
‘Job-related psychological effects’. This construct is further
divided into four variables; 'Distress', 'Information Anxiety',
'Creativity' and 'Sharing of tacit knowledge'. This implies that
increased use of a Knowledge Database can lead to increased distress
and
information anxiety, and decreased creativity and sharing of tacit
knowledge. The reason for choosing these negative effects as variables,
is based on the definition of what a Knowledge Database is and does,
and the description of a KW. These two seen together with the field of
organization psychology, have initiated the described psychological
effects as possible negative ones.
It can be argued that ‘Innovation’ could substitute ‘Creativity’,
because it gives a broader view, which includes the organization.
However, since the chapter is focusing on the individual level, it is
found that creativity is better suited. The individuals are doing their
work within the already determined frames, without changing the
processes of the company. Creativity means finding good solutions for
different and dynamic problems; while innovation means that the firm is
changing their already fixed processes, procedures or systems. This
will reflect another dimension. It is also important to notice that the
type of stress included in the research model is of negative nature.
The research model presented could be viewed as an ‘A-B-C model’ where
construct ‘C’ being ‘Job-performance’. ‘Job-performance’ can be defined
as an assessment of the amount, quality, and value of an individual’s
contribution to the workplace based on performance of assigned
responsibilities (Greenberg & Baron 2003). It is affected by the
‘Job-related psychological effects’, which occur as a consequence of
using the Knowledge Database. Our focus will be on the two constructs;
‘Use of a Knowledge Database’ and ‘Job-related psychological effects’
shown in the research model.
Based on the arguments presented the following four hypotheses are
created:
H1: Increased use of Knowledge Databases leads to increased distress
H2: Increased use of Knowledge Databases leads to increased information
anxiety
H3: Increased use of Knowledge Databases leads to decreased creativity
H4: Increased use of Knowledge Databases leads to decreased sharing of
tacit knowledge
Figure 1: Graphical Research Model
This section will give a brief discussion of the level of investigation and the method used. In addition there will be a sub-section presenting the creation and pre-/post-preparation of the interview guide.
The unit of analysis is to be focused at the
individual level. The KM processes start at the individual level, and
affect the organizational level through the group level. The
individuals are then influencing the overall organizational
performance. Organizations can therefore benefit from the use of an IS,
such as a Knowledge Database, and increase returns through their
expanding organizational knowledge based on the individual growth of
their employees. (Sabherwal & Fernandez 2003; DeLone & McLean
1992) It is therefore important to address the issue concerning the
individual level in order to cope with any difficulties at an early
stage.
When investigating the negative effects of KMS use the population is
all employees using a system that can be defined as a KMS in their
daily work. The sample in this research consists of two lawyers from a
law
firm in Oslo.
There are two different approaches to conducting
research, inductive and deductive. The inductive approach is based on
the principle of developing theory after the data have been collected
(Saunders et al. 2003), as opposed to the deductive approach which is
used when a clear theoretical position is developed prior to the
collection of data (Saunders et al. 2003). This chapter will focus on
using the inductive approach since the theory within this field is
somewhat limited.
There are three main types of research design. These are causal,
explorative and descriptive (Ghauri & Grønhaug 2002). The
research design used in this chapter is the explorative design, which
is
chosen because the theory area of research is limited, and the research
aims at finding the relationship between use of KMS and negative
effects related to the individual within an organization. (Ghauri &
Grønhaug 2002)
The two main methods for collecting data are qualitative or
quantitative methods. The qualitative methods put the emphasis on
understanding, where as the quantitative method emphasises testing and
verification. (Ghauri & Grønhaug 2002) The research in this
chapter is conducted based on a qualitative method. This chapter uses
the
semi-structured interview in order to capture most of the various
nuances in the variables. Through such interviews it is possible to
gain the in-depth understanding that is required to answer the research
question. In addition, the theory of the area is somewhat incomplete,
which makes it difficult to find specific theory to build this research
on. Due to this, it is important to use a research method that will
allow the researcher to be flexible and somewhat unstructured, which is
possible when using a qualitative method like semi-structured
interviews.
The interview guide is based on our choice of
method and theory. The structure and formulation of the interviews is
based on literature by Larsen (2004) and Ghauri & Grønhaug
(2002). The topics examined in the interview guide are compound of
theories presented in the theory chapter. Questions measuring the use
of the database are inspired by Doll & Torkzadeh (1998). The others
were created based on literature that explains and gives definitions of
the psychological phenomena.
The interview guide is meant as a model for the interview, but it
allows for changes in the structure and order of questions during the
interviews. Our interview-guide is presented in one interviewer version
and one interviewee version (Appendix
3). The reason is to avoid the interviewees to give answers that
are led by what is being investigated. Therefore, the version shown to
the interviewees was without sections; that showed what the questions
were measuring, and without definition other than those necessary to
understand the questions. The interview guide was further developed in
English and translated into Norwegian. To control the intended meaning
and formulation of the questions in the interview guide, it has also
been translated back to English by another person. The interview guide
was sent to the interview objects approximately one week before
conducting the interviews. A transcript of the interview was sent to
the interviewees for review, corrections and additional comments. The
findings, with additional information from the interview, were
implemented into the interview guide (Appendix
4).
The results were analyzed using protocol analysis (Appendix
5). A matrix including all questions about the constructs in the
model was created, where an explanation regarding the classification
High-Medium-Low was presented. This was then used when the researchers
separately graded the questions. A matrix with all the answers was
filled out. Here it was also taken into account whether there was an
agreement or not. Finally, an analysis of reliability between the items
representing a variable was created, based on the result in the other
matrix.
This section gives a description of the firm and industry, and an analysis of the four hypotheses.
The chosen industry in this research is the law
industry. The industry consists of many actors, both large and small in
scale and either specialized towards the private or the government
market. Since approximately 2000, there has been a shift in the demands
from the clients, which has resulted in new structures. This shift has
been from generalization towards specialization within law firms,
mainly because of increased competition. Law firms have to be on top of
their game, and be able to offer lawyers who know fields in depth. This
trend is especially seen in large cities.
Lawyers are highly individualistic and competitive in nature. They
often prefer to work alone and are very independent. As a lawyer, it is
also extremely important to be creative and look for practical
solutions. Lawyers can therefore be seen as a type of knowledge
workers.
They depend on knowledge to create the best possible solution. This
knowledge might either be available knowledge; as in a database, or
tacit knowledge; the personal knowledge one possesses.
This research was conducted in a law firm located in Oslo. It is a full
service law firm with offices in Copenhagen, Oslo, Stockholm
and Bergen. The company is one of the largest firms in the Nordic
region. (Law firm 2004) The office in Oslo consists of 124 employees.
The interview objects chosen are two Senior Associates in the firm. The
objects are chosen because they are typical knowledge workers, and they
actively use a Knowledge Database, called “Docu-share”, in their
everyday work. Since the research is focused on the individual level
the two candidates have been chosen from the same firm.
In this section there will be an analysis of use related to the Knowledge Database and the four hypotheses stated.
The dependent variable “Use” was measured by “Use
of the Knowledge Database & Degree of Use” and “Perceived
Usefulness & User satisfaction”. Through investigating the results,
provided in the tables below, one can observe that both have an average
of medium. The questions regarding how much they use the database being
a little towards low, while the satisfaction being a little towards
high. It is therefore possible to conclude that they are using the
Knowledge Database and is familiar with it. They think it is rather
easy to use, useful, and have an impact on daily work procedures. By
establishing that the database actually is used and have an impact on
work, it is further possible to investigate the four hypotheses.
~ Increased use of Knowledge Databases leads to
increased distress ~
This hypothesis is focusing on if using a Knowledge Database leads to
more distress in the sense that you may be forced to provide/create
information to the database. This was through the interview
investigated by asking questions about distress in the workplace and
changes in occupational demands in relation to making the knowledge
available to other employees in the Knowledge Database. By observing
the matrix presenting the results regarding these questions, it is
clear that the results are somewhat vague. This is due to the fact that
there are no relation with the distress the respondents felt/did not
feel, and the process of making their knowledge available in the
database. One of the interviewees felt that there are no distress
related
to the database and no changes in demands in making the knowledge
available. The other also felt no distress related to the database, but
thought there had been some changes in demands. The company has few
routines regarding updating the Knowledge Database and it is then not a
regular task. This creates a weak ground to build any theory and
conclusions on.
One of the respondents felt some distress related to work, but this is
more from clients and outside the organization, than inside. The other
did not feel stressed at all. One had experienced some small changes in
workload, because making knowledge available in the database becomes an
additional task to regular work. But because these demands are
forwarded so seldom, it does not create any distress. The other did not
experience any change in workload but said: “It is an additional
job.
This can be a pain and is not a very grateful assignment. This can be
an obstacle for including more information in the database. If it was
released or granted more time for this within the organization more
would probably be done in this field.”
Hypothesis 1 is in this case not supported, but the results are weak
and hardly state anything – nor supported or rejected.
~ Increased use of Knowledge Databases leads to
increased information anxiety ~
This hypothesis is focusing on if using the Knowledge Database leads to
increased information anxiety, in relation to the Knowledge Database
having too much and unorganized information. This was through the
interview investigated by asking questions about changes in workload as
a consequence of using the database and difficulties surrounding the
search for relevant information and limiting the search. The matrix
below presents the results and findings, showing that there is no
information anxiety related to the use of the database. Both feel the
workload is unchanged and that the database has made the job-process
more efficient. They also perceive the database as simple and easy to
use, with rather too little than too much information in it.
Hypothesis 2 is not supported.
~ Increased use of Knowledge Databases leads to
decreased creativity ~
This hypothesis is focusing on if using the Knowledge Database leads to
decreased creativity in the sense that employees become less creative
when using the database. Through the interview this was investigated by
asking questions about creativity in their daily work, and in what way
the Knowledge Database has had an impact on the creativity. By
analysing the results on these questions, it is clear that creativity
is very important in a lawyer’s daily work. Both respondents emphasise
the importance of this aspect in their work. Creativity is important
because as a lawyer one meets practical problems that require practical
solutions, which has a lot to do with being creative. There are several
ways to find a suitable answer, and it is important to find the best
solution for that particular client. This, in order to make the client
satisfied. Within their company they are actively being encouraged to
creativity, to take initiative and search for different solutions. One
of the respondents said: “Too succeed as a lawyer you have to be able
to be creative” .
Both respondents claim that the use of their Knowledge Database might
to some degree limit the employees’ creativity. The Knowledge Database
is not used to find good ideas and suggestions for solutions in
different situations, but is rather used when the solution is apparent.
It might be used as a pillow for some. By using the information
available in the database, some might be using it blindly, and not
adapting it to the client in the best possible way. Some might also
feel obligated to use the templates in the Knowledge Database, and
forget
to add their own personal thoughts, making them miss-target the best
solutions for the client.
The Knowledge Database has had high impact on the lawyers’ creativity, making the creativity decrease in their daily work. This results in hypothesis 3 being accepted. The increased use of Knowledge Databases leads to a decrease in creativity.
~ Increased use of Knowledge Databases leads to
decreased sharing of tacit knowledge ~
This hypothesis is focusing on if using the Knowledge Database leads to
less sharing of tacit knowledge within the company. This was through
the interview investigated by asking questions concerning changes in
communication between co-workers after implementation of the Knowledge
Database, how the respondent perceived tacit knowledge as a resource in
their work, and how the company culture was regarding helping each
other and in what way the Knowledge Database played a role in this
process. By analysing the results of these questions it is shown that
the implementation of the Knowledge Database has had an impact on the
communication within the organization. One of the respondents indicates
that there have been large changes, while the other respondent
indicates moderate changes in communication. As a result of
implementing the Knowledge Database the communication between
co-workers has decreased. Employees are encouraged to mainly use the
Knowledge Database, and therefore do not ask their co-workers for help
and advice the same way they used to. Both respondents emphasise the
importance of tacit knowledge as a resource in their daily work, and
that this type of knowledge is best communicated through communication
between co-workers. The respondents both agree on the fact that the
Knowledge Database plays an important role in the process of sharing
knowledge within the company. The culture in the company for sharing
knowledge and helping each other is by one of the respondents described
as very good, and there is a willingness to help. The second respondent
indicates that the culture for helping each other is not very good,
especially among the young employees who have a tendency of viewing
each other as rivals.
The use of the Knowledge Database has had high impact on the lawyers’ sharing of tacit knowledge, making it decrease. This results in hypothesis 4 being accepted. The increased use of Knowledge Databases leads to a decrease in sharing of tacit knowledge.
The research question of this chapter was as the following; “Will increased use of Knowledge Databases lead to an increase in unwanted job-related psychological effects?” The two lawyers interviewed in this research uses the Knowledge Database from several times a week to once every second week. The Knowledge Database is stated to be important in a lawyer's work, and helpful in providing the lawyers with templates and standards for use in every day routines. This section will discuss the findings in the four hypotheses, the research related to method and refinement of research model based on new and eye-opening information retrieved in the interviews.
The hypothesis had no clear answer regarding if the
Knowledge Database created increased distress. The reason for this
result was
that the stress they felt/not felt did not have any thing to do with
the use of the database. They both confirmed that they did not feel
distress in relation to contributing with information to the database,
and they slightly disagreed whether the database facilitated a change
in demand for sharing knowledge. Due to this, there was no clear result
that the database inhibits or facilitates distress in relation to
creating documents for sharing in the database.
The company has few routines when it comes to updating and providing
the database with new information. As stated by one of the
interviewees: “The information in the database is a bit old and not
updated. The oldest contribution seen is 9 years old, but also a lot
from 2000.” In relation to creating documents for the
database, it was further stated: “You are informed if it is desired
that you make a template, and this is usually not urgent. But it is an
ungrateful job, because it usually comes in addition to everything else
that has to be done.” But as both stated, this did not
happen often, and thereby did not feel stressful. Repeatedly exchanging
information with others can, as Greenberg & Baron (2003) has
stated, be a great stress factor. The company investigated in this
chapter had no routines in repeated information exchanges, and this
made
the hypothesis difficult to investigate.
The hypothesis was rejected regarding if using the
Knowledge Database could lead to Information Anxiety. Both respondents
felt that there were rather too little than too much information in the
database, and its simplicity makes it easy to use. They had no problems
in finding what they searched for. It is due to mention that the
database has a very simple structure, with no search tool. It is
categorized in groups and sub-groups, which one must click in to.
One interesting finding, which made the hypothesis relevant for further
investigation, was that one respondent also used Lovdata, which is a
database for law sentences and preparations. Here the interviewee
experienced great difficulties in finding the information needed
because of a nonuser-friendly searching tool and the amount of
information available – the black hole between data and knowledge
(Wurman 1989) becomes too large. This created so large frustrations
that the person looked for information elsewhere. “It is not very
user
friendly, so sometimes the information has to be located from other
databases.” This implies that the result heavily relies
on the database being used and that there are databases out there which
might create information anxiety and information overload.
The hypothesis was accepted regarding if using the
Knowledge Database could lead to decrease in the creativity. Both
respondents stated that using the Knowledge Database to some degree
made the creativity decrease. “It can in some cases inhibit
creativity,
because one might get blinded by a standard…”. Both
respondents clearly stated that the creativity is very important in
their daily work as lawyers. A decrease in the creativity among the
lawyers may result in creating bad solutions for the client. “The
database can become a pillow for some, and thereby limit the
creativity, and you might feel constrained by the frames provided by
the database. It may result in creating bad solutions”.
According to Reiter-Palmon & Illies (2004) employees must define
and construct a problem, search and retrieve problem-relevant
information, and generate and evaluate a diverse set of alternative
solutions. As stated by the respondents the Knowledge Database can
create a threat to the creativity by not forcing the lawyers to
evaluate a set of different solutions, but rather making it possible to
choose a solution already available in the Knowledge Database. When
managing KWs it is important to keep in mind that they need to be
energized intellectually (Gottschalk 2002b) in order to maintain
creativity. In this relation it can be worth asking if a Knowledge
Database can facilitate this, or if it is an obstacle that in the worst
case can destroy creativity.
The hypothesis stating that increased use of a
Knowledge Database leads to decreased sharing of tacit knowledge was
accepted. Both respondents stated that there has been a significant
change in the communication between co-workers after the implementation
of the Knowledge Database and that the tacit knowledge is very
important. An decrease in communication between co-workers result in
less sharing of tacit knowledge, due to the fact that the tacit
knowledge is difficult to capture in a database.
Less communication between co-workers has indirectly led to less
sharing of tacit knowledge. As stated by one of the respondents: “(…)I
believe it is more difficult to get access to personal knowledge today”.
When sharing knowledge within an organization it is
necessary to convert the tacit knowledge into explicit knowledge, and
this can be seen as finding a way to express the inexpressible (Nonaka
1991). The fact that tacit knowledge is best shared through discussions
and person-to-person interaction makes it extremely difficult to
capture in a Knowledge Database (Gartner 2004). This underpins the
results found in the research that less communication between
co-workers, due to information being available in a Knowledge Database,
leads to a significant loss in sharing of tacit knowledge. It can be
discussed whether a Knowledge Database is a good tool for capturing
tacit knowledge. Other KMS’s, which includes discussion forums and
collaboration applications, might provide a better solution for sharing
tacit knowledge.
The research has been conducted through an
inductive approach, which was well suited. The goal is to gain new
knowledge about the field and topic of interest. The research was
conducted through two semi-structured interviews. The flexile and
unstructured way of doing these interviews was well suited for this
particular research, due to the fact that the interview objects were
very responsive and there was no need for more structure and
constraints. The semi-structured approach was also very constructive
because the interviewers were inexperienced and the need for some
structure and freedom was required.
The researchers had divided the interview into main sections related to
the research model, which was helpful when doing the analysis. The
researchers also made two versions of the interview guide, so that the
interviewees would not be influenced by the headings and the framing of
the questions. The relevant interview guide was forwarded to the
interviewees a week before the interview, in order to make the
respondents prepared for the interviews. This seemed effective; the
interviewees had then read through the guide and were well informed
when the interview took place. Due to the fact that the interview
objects were answering the questions under each section unknowing of
the overall implication, the researchers got the unbiased answered
intended.
The research conducted resulted in both rejections
and confirmations of the forwarded hypotheses. Due to the fact that two
hypotheses in the model were confirmed, the research question can be
argued to be valid and highly relevant. The last two hypotheses were in
a way rejected. This was due to the fact that the company lacked
routines concerning creating and adding documents into the knowledge
database, as well as the size of the database. Based on underlying
theory and the fact that the results might have been different if the
selected company had been more dedicated towards using and updating a
Knowledge Database, the original research model is kept as basis for
following refinements.
Through the interviews the researchers became aware of additional
variables that might be relevant to investigate related to negative
effects of Knowledge Database use (Appendix 4 – Additional
information). The research model can then be extended to include two
more variables; increased internal competition and decreased normative
commitment. The variable ‘internal competition’ was found based on the
interviewees’ comments about a change in company culture after the
implementation of the database. The implementation has lead to a
greater unwillingness to share knowledge and heavier internal
competition based on policy of using the database before/instead of
asking others. This is also rooted in a lawyer’s individualistic
nature, and the competitive environment.
The variable ‘normative commitment’ builds on information which
indicates that the willingness to help each other is reduced after the
implementation of the Knowledge Database. It has become the company’s
policy to use that Knowledge Database before/instead of asking a
colleague. Normative commitment can be defined as the commitment an
employee has for the organization based on the cohesiveness and culture
the employees share together (Greenberg & Baron 2003). The social
networks and groups are affected as a result, which implies that there
is less interactions between colleagues. This effect might be important
to the employees which in the end indirectly affects the organizational
performance. These new variables are both more related to a group level
than the other hypotheses; which were concentrated towards the
individual level, and will provide new and interesting areas for
investigation.
This creates the following two hypotheses:
H5: Increased use of Knowledge Databases leads to increased internal
competition
H6: Increased use of Knowledge Databases leads to decreased normative
commitment.
Figure 2: Extended research model including the additional variables from interviews
This section will present some thoughts regarding theoretical and practical implications this chapter provides.
The research done in this chapter might be viewed
as a tiny step in the direction of mapping the negative effects that
might occur when using a KMS. The hypotheses were both supported and
rejected. Although findings implied that all hypotheses should be
objects to more thorough investigation before any conclusions can be
made. As to creativity and sharing of tacit knowledge, it is clear
indications of Knowledge Database inhibiting this process. This is not
necessarily generalizable, due to the small sample size, but can be
viewed as an indicator that it is worth paying attention to in future
literature. As to distress and information anxiety, the findings were
limited, and the results show no directly linkage to the use of the
Knowledge Database. Though, information given during the interviews
implied that the picture is somewhat more nuanced then the results
might imply. Due to this, further investigation with other Knowledge
Databases is necessary in order to create solid theory.
The results imply that future research should build upon the theory
presented in this chapter, and expand upon it in order to find other
negative effects than those investigated here. The chapter states two
new hypotheses that can be the base for further investigation, in
addition to the four already tested. One alternative might be to do
quantitative research with a large number of respondents, using several
Knowledge Databases. This will provide answers about what kind of
Knowledge Databases that has the negative effects, which will
ultimately give indication of what to do to avoid them.
As the results of this research indicate that there
are unwanted negative psychological effects related to the use of a
Knowledge Database, it is argued that the practical implications of
this chapter support the need of more awareness when implementing a
Knowledge Management System. More thorough investigation of negative
outcomes that might occur is needed. Recent literature (Schultze &
Leidner 2002) has stated the lack of focus on the negative effects that
can occur when implementing an IS, in this canse a KMS. This awareness
should also be in the minds of those making the decision of
implementing the system. By knowing about the side effects the system
might provide, it is possible to eliminate them before they even occur.
This will give the system a much better chance of succeeding in the
task initially planned.
It is in this research found that the use of Knowledge Databases leads
to a decrease in creativity. As with Knowledge Workers, in this case
lawyers, it was stated during the interview that creativity is vital to
be able to succeed. Creativity can be viewed as one of the corner
stones in the process of case handling, and to have a system that
decreases this creativity is highly negative. This will reduce the
quality of the work, which in the end will have an impact on the
organizations overall performance. To avoid loss in creativity, it is
important that the management guides their employees in using the
database in a best possible way. It is also important that the database
is viewed as a support-tool and not a decision system.
It was also found that the use of Knowledge Databases leads to a
decrease in the sharing of tacit knowledge. Tacit knowledge is a
valuable resource for the knowledge workers and difficult to capture in
an IS. To view the organization as a body of knowledge, and facilitate
a sharing of knowledge within the organization can be essential to
sustain a competitive advantage in the industry. To have a database
that inhibits this sharing, will have an impact on job performance, the
learning curve that each employee possesses and the culture in the
company. To reduce the loss in sharing of tacit knowledge, other KMSs
that in at better way capture this can be implemented, e.g. discussion
forums. But there is a problem in the legal industry concerning the
individualistic nature of the lawyers and the procedures with billing
their customers that prevents pulling other lawyers into the case,
which has to be taken into consideration. No clear answers are yet to
be found in how this should be addressed. Lawyers are due to this a
special type of KW in the sense that they create a more complex picture
when it comes to the use of a KMS than other KW might do.
The research conducted in the law firm has its
limitations. The research is based on the fact that lawyers are typical
knowledge workers and they use Knowledge Databases in their daily
routines. Our two interview objects from the law firm are randomly
picked
out from the firm’s 124 employees in the Oslo office. The first
limitation is that the sample only consists of two subjects. The two
interview objects work within the same firm and they are approximately
at the same level in the organization. This implies that the results
are not necessarily generalizable. To conduct the interviews with a
larger sample
from the firm, divided across different levels, at different age, and
with different degrees of experience in order to get a broader view
would ground a stronger result.
The law firm has been using the Knowledge Database since 1996-97 and
the
two interview objects are both been hired in the time after the
implementation of the Knowledge Database. This leads to the second
limitation. The interviewees have not been a part of the transition
process from the old system to the new systems which includes the
currently used Knowledge Database. This implies that they do not have
the total knowledge required to answer questions related to
before/after-situations. It will be necessary to interview employees
that have been with the firm during this transition process, in order
to better investigate whether there have been some changes in the work
routines and the communication.
The third limitation is concerning the actual routines regarding
updating the Knowledge Database and adding contributions to it. In the
law firm there are no frequently routines related to updating and
contributing information. The employees are not required to contribute
information to the Knowledge Database, but when they do they are often
told by the head of the department or a partner that sees the need for
it. This is done rather seldom and therefore the employees do not feel
they are using valuable time on it. If the firm had had routines for
updating and adding information into the Knowledge Database the
research would be better suited.
Fourthly, a relevant issue is the age differences and different titles
in the firm. In a large firm there are a lot of
employees divided across several age groups and they possess different
titles. The partners are usually older than the associates, and it is
also known that the older employees use the Knowledge Database less
than their younger colleagues. This might be because the older ones do
not see the full potential of the Knowledge Database and its use, or
that they actually see the drawbacks it might create. They rely more on
their experience and tacit knowledge. To broaden the picture, it would
be interesting to interview employees of higher age and position in the
firm, since the objects used in this chapter are relatively young and
does not have the much experience.
The purpose of this chapter was to investigate if
increased use of Knowledge Databases leads to an increase in unwanted
job-related psychological effects. This was investigated through
semi-structured interviews with two lawyers. This resulted in two
hypotheses being verified, which imply that use of a Knowledge Database
leads to less creativity and sharing of tacit knowledge. In the two
other hypotheses regarding distress and information anxiety little was
found. Due to this there were indications that the phenomena need
further investigation in order to state that the hypotheses can be
rejected or confirmed. In relation to this, the main research question
has been verified – use of Knowledge Databases leads to an increase in
unwanted job-related psychological effects.
The research done in this chapter has in many ways shown that KMS truly
can be a double-edged sword. Through the hypotheses it is found that a
Knowledge Management System creates unwanted, negative,
job-related effects, which has to be taken into account in order to
make the system work and serve as efficient and useful as intended. It
is therefore important for researchers to focus on the negative aspects
in order to develop a stronger theoretical base that includes both
favourable and unfavourable consequences of knowledge and its
management. This research can be beneficial for managers and alike that
is currently using, or considering using, such a Knowledge Database.
This research triggers the attention at the non-beneficial effects that
should have gotten more attention when introducing a KMS. The negative
effects may be worth paying attention to in order to prevent a
system-failure – when minor adjustment could have avoided it.
This chapter can serve as a tiny step towards mapping the negative
consequences that might occur in relation with using a KMS. The results
indicate that further research has to be done in order to build some
solid theory upon it. As to creativity and sharing of tacit knowledge,
it is found clear indications that a Knowledge Database inhibits that.
Distress and information anxiety was in this research found to be not
directly related to the use of the database. Though, information given
during the interviews indicates that the picture is not as black and
white as it might appear.
Future research should focus on finding other negative effects that
might have an impact, as well as build and expand on the research model
presented in this chapter. One alternative might be to do quantitative
research with a large number of respondents, using different Knowledge
Databases, to verify if the hypotheses presented here are
generalizable. Other types of KMS’s should also be included, to make
the picture as complete as possible. These may also have other types of
effects than the ones investigated here.
Appendix 1 - DeLone and
McLean’s (1992) “Model of IS Success”
Appendix 2 - Seddon’s
(1997) “Respecified version of DeLone and McLean’s
(1992) Model of IS Success”
Appendix 3 - Interview
Guide
Appendix 4 - Interview
Guide including answers from interviewees
Appendix 5 - Protocol
Analysis
Alavi, M. and Leidner, D. E. (2001) "Review:
Knowledge Management and Knowledge Management Systems: Conceptual
foundations and research issues", MIS Quarterly, Vol. 25(1),
pp. 107-136
Amabile, T. M. (1983) "The social psychology of creativity: A
componential conceptualization", Journal of Personality &
Social Psychology, Vol. 45, pp. 357–376
Anderson, C. A., Arnoult, L. H. (1989) "An Examination of Perceived
Control, Humor, Irrational Beliefs, and Positive Stress as Moderators
of the Relation Between Negative Stress and Health", Basic &
Applied Social Psychology, Vol. 10(2), pp. 101-118, 17p; (AN
7299121)
Barney, J.B (1991) "Firm resources and sustained competitive
advantage", Journal of Management, Vol. 17(1), pp. 99-120
Barney, J.B (2001) “Is the resource-based ‘view’ a useful
perspective for strategic management research? Yes”, Academy of
Management Review, Vol 26(1), pp. 41-56
Barney, J.B (2002) Gaining and Sustaining Competitive Advantage,
2nd edition, New Jersey: Prentice Hall
Business Source Premier (2004), http://www.bi.no/bibliotek
(06.09.04), Literature review on Knowledge Management Systems and
effects
Caldwell, F.,Linden, A., Miklovic, D., Morello, D., Knox, R., Logan,
D., Grey, M., Gilbert, M., Harris, K., Andrews, W., Lundy, J., Hayward,
S., Bell, M., Latham, L., Shegda, K. (2003) “Hype Cycle for Knowledge
Management 2003”, Strategic Analysis Report, June 2003
DeLone, W. H. and McLean, E. R. (1992) “Information Systems Success:
The Quest for the Dependent Variable”, Information Systems Research,
Vol. 3(1), pp. 60-95
Doll, W. J. and Torkzadeh, G. (1988) “The measurement of end-user
computing satisfaction”, MIS Quarterly, Vol. 12(2), pp. 258-275
Fagan, M. H., Neill, S. and Woolridge, B. R. (2003-04) “An empirical
investigation into the relationship between computer self-efficacy,
anxiety experience, support and usage”, Journal of Computer
Information Systems, Vol. 44(2), pp. 95-105
Fahey, L. and Prusak, L. (1998) “The Eleven Deadliest Sins of Knowledge
Management”, California Management Review, Vol. 40(3), pp.
265-276
Ghauri, P. & Grønhaug, K. (2002) Research Methods in
Business Studies, 2nd edition. USA: Prentice Hall
Gartner (2004) “Gartner’s Glossary of Terms Used for the Knowledge
Workplace 2004: Update”, Strategic Analysis Report, April 2004
Gottschalk, P. (2002a) It-strategi, Norway: Fagbokforlaget
Gottschalk, P. (2002b) Knowledge Management through Information
Technology, Norway: Fagbokforlaget
Greenberg, J. and Barone, R. A. (2003) Behaviour in Organizations:
Understanding and managing the Human Side of Work, New Jersey:
Prentice Hall
Hackbarth, G. (1998) “The impact of Organizational Memory on IT
Systems”, Proceedings of the Fourth Americas Conference on
Information Systems
Hitt, M.A., Bierman, L., Shimizu, K. And Kochhar, R. (2001) ”Direct and
moderating effects of human capital on strategy and performance in
professional service firms: a resource-based perspective”, Academy
of Management Journal, Vol. 44(1), pp. 13-28
Interviewee 1 (2004) Interview 27.10.04, Senior Associate
Interviewee 2 (2004) Interview 27.10.04, Senior Associate
Johnston, D. (2004) Undoing Stress, http://www.lessons4living.com/stress%20topics.htm
(08.10.04)
Larsen, T. J. (1999) “Line Managers’ Supervision of the Development and
Use of Information Systems (IS): Administration through Planning and
Control of IS Effects,” in Khosrowpour, M., (Ed.), Managing Information
Technology Resources in Organizations in the Next Millennium, Proceedings
of the 1999 Information Resources Management Association International
Conference, Hersey, PA, May 16-16, pp. 853-860
Larsen, T. J. (2004) Lectures in GRA 6645 MIS Research Topics, Fall
term
Laudon, K. C. and Laudon, J. P. (2003) Essentials of Management
Information Systems: Organization and Technology in the Networked
Enterprise, 5th Edition, USA: Prentice-Hall
Law firm (2004) “About XXX”, (The website is not included due to
anonymity)
(28.10.04)
McNish, M. (2002) “Guidelines for managing change: A study of their
effects on their implementation of new information technology projects
in organisations”, Journal of Change Management, Vol. 2(3), pp.
201-212
Nonaka, I. (1991) “The Knowledge creating Company”, Harvard
Business Review, November-December, pp. 96-104
Perrewé, P. L., Zellars, K. L., Ferris, G. R., Rossi, A. M.,
Kacmar, C. J. and Ralston, D. A. (2004) "Neutralizing Job Stressors:
Political Skill as an Antidote to The Dysfunctional Consequences of
Role Conflict", Academy of Management Journal, Vol. 47(1), pp.
141-153
Reiter-Palmon, R. and Illies, J. J. (2004) "Leadership and creativity:
Understanding leadership from a creative problem-solving perspective", Leadership
Quarterly, Vol. 15(1), pp. 55-78
Sabherwal, R. and Fernandez, I. B. (2003) "An Empirical Study of the
Effect of Knowledge Management Processes at Individual, Group and
Organizational Levels", Decision Sciences, Vol. 34(2), pp.
225-260
Sauer, C. (1993) Why information Systems Fail: A case study
approach, Henley-on-Thames: Alfred Waller
Saunders, M., Lewis, P. and Thornhill, A. (2003) Research Methods
for Business Students, Edinburgh: Prentice Hall.
Schultze, U. and Leidner, D. E. (2002) "Studying Knowledge Management
in Information Systems Research: Discourses and Theoretical
Assumptions", MIS Quarterly, Vol. 26(3), pp. 213-242
Schwarz, G. M. and Brock, D. M. (1998) “Waving hello or waving
good-bye? Organizational Change in the information Age”, International
Journal of Organizational Analysis, Vol. 6(1), pp. 65-91
Seddon, P. B. (1997) “A respecification and extension of the DeLone and
McLean Model of IS success”, Information Systems Research, Vol.
8(3), pp. 240-253
Venkatesh, V., Morris, M. G., Davis, G. B. and Davis, F. D. (2003)
“User acceptance of information technology: toward a unified view”, MIS
Quarterly, Vol. 27(3), pp. 425-478
Von Krogh, G. (1998) “Care in Knowledge Creation”, California
Management Review, Vol. 40(3), pp. 133-153
Wurman, R. S. (1989) Information Anxiety, New York: Doubleday
& Co Inc
This paper focuses on the perception of value of a Business Intelligence (BI) system and the purpose is to put focus on the use of KPI's in the corporate setting. A vendor of a BI system and a customer of the vendor are questioned about their perception of the BI system. The results of this paper states that there is only a few or no Key Performance Indicators (KPI) in use today, but the use is beginning to accumulate. The studied likeness in perception is very high and indicates realistic expectations to the system.
Business Intelligence (BI) systems are getting extensively used in companies today and according to media the use of Key Performance Indicators (KPI) are important in this system. The focus on this subject is because the authors want to study how the KPI's are used in companies and what kind of KPI's that are used since there are so much focus on this system in media today.
BI systems are getting a lot of focus at the time being due to an increasingly need for information. As part of this more vendors are selling their software as BI systems. These vendors offer systems which can gather and analyse information from different locations and present results in an easy to use way. The information is often presented in a digital dashboard accessed by business managers. However the dashboard consists of several measurements from the analysis. These measurements are refereed to as KPI's. This paper uses the use of KPI’s as a key measurement of BI use. More KPI’s used will reflect more BI use. An interesting aspect here is whether the perception of the value of KPI’s is influencing the use of BI system. A difference in perception between the seller of BI systems and user of these systems could have influence on the actual use.
Looking for information about BI systems is not an easy matter since this is a new concept and is not been used too much academic. For the authors of this papers view there is a lack of information in this area. More research has been done into the field of Online Analytical Processing (OLAP) and decision support systems (DSS), which is quite similar to BI systems, but is not as extensive in use. BI systems can also be referred to as the sets of tools and applications that query the OLAP data and provide reports and information to the enterprise decision makers. The studies done within the field of BI are mainly studies which investigate the strategic value added by using BI systems (Williams, 2004, Bauer, 2004a, Bauer, 2004b), the cost and benefits of BI systems (Barberg, 2004) and the visibility of KPI’s in BI systems (Bauer, 2004a, Bauer, 2004b, Carman and Conrad, 2000, Reh, 2004). More general studies about IT project successes, gap of expectations in ERP systems and its effect on customer satisfaction (se chapter 2 by Eklöf, Spieler and Tukh) and the strategic value of IT are among a broad range of studies done in the IT area. This article is going to look into how BI systems (a type of IT system) are perceived by vendors and users to investigate a value gap. This value gap can tell about the perceived value of BI systems and whether it affects how the systems are used in organizations reflected by the use of KPI.
The main contribution of this paper is to show that perceived value of BI systems affect the use of these systems. The use in this paper is based on the organizations perception of the value of KPI’s and their use of these measurements. A study which is also looking into customer perception (perceived expectations) and how overselling a software may cause less customer satisfaction is studied in this web book (chapter 2) by Eklöf, Spieler and Tukh. They view the overselling of software versus the customer satisfaction, while this study looks at the perception gap and how this affects the perception of BI value defined by the use of the BI system.
The content of this paper will be presented in the headings theory, research design and method, analysis, discussion, conclusions, references and appendixes.
The use of KPI as a measurement of BI use can be discussed, but the authors of this paper view the KPI as a very important part of a BI system. Using the number of KPI’s to evaluate BI system use will explain how extensive the BI systems are. If this information is combined with the information of number of KPI’s in the beginning of a project and the number of KPI’s after having used the BI systems for a while that may indicate the perceived value of the BI system.
What are the relationships between the perceived value of BI systems, the likeness in perception between vendor and customer and the business manager’s use of KPI’s?
The use of KPI’s in the BI system indicates a certain amount of use of KPI’s. The authors initial though was that companies start using a specific set of KPI’s and during use realizes that there are other KPI needs. This need is then implemented in the form of more KPI’s. The number of KPI’s is indicating the perceived value of BI systems in this case. Measurements other then KPI’s are not taken into account here as to limit the study. This leads to the first proposition.
Does increasing perceived value of the BI system increase the business manager’s use of KPI’s?
As the chapter (2) by Eklöf, Spieler and Tukh discusses, overselling the software is probably not a good strategy. The focus in this paper is to look into how the value of a BI system changes from the initial use and during the time of use. The value is described by the number of KPI’s used. If the value is perceived the same then there has been no overselling or the BI systems have delivered the expected value. This is probably the ideal situation since it does not decrease the customer satisfaction. This leads to the second proposition.
Does an increasing likeness in perception of value of the BI system between vendor and customer increase the business manager’s use of KPI’s?
The authors of this paper believe that the use of KPI’s increases as the perceived value of the BI system increases. There is therefore an expected positive outcome of the first proposition. The second proposition is also believed to have a positive impact and the expected outcome here is that there is a gap in the perception, which influences the use of KPI. The expectations of both these propositions are dependent on the use of BI systems and KPI in organizations.
A negative outcome of the first proposition will indicate that the perception of system value does not contribute to the use of KPI. A negative result will also imply discussion of why organizations do not use KPI’s based on the perceptions of the value and what influences the KPI’s if not the perceived value. A negative result could be a result of the wrong object of interview, which does not perceive value of KPI’s.
A negative outcome in the second proposition will indicate that the perception of value of a vendor and a customer does not need to be the same. Whether they perceive the same result or not is not influencing the use of KPI’s. This means that overselling the software or one of the parties having a better perception of value of the system does not influence the use of KPI in an organization.
No use of KPI in the companies studied will lead to a discussion of why they are not used, if this could be coincidental and what the impact is of these KPI’s not are being used. An alternative measurement will be discussed if this is discovered. This measurement will then not be a KPI, but more regular key figures.
For business the study will try to reveal how important the perceived value of a system is for its use and how important it is to not oversell the system. A balanced perception of value increases use of KPI’s in organizations. Other important impacts of the study are to show how extensively KPI’s are used in organizations or if this is just a buzzword, how the perception of a system is from both the vendor and customer point of view and how the perception of value has changed during usage of the BI system.
For researchers the appliance of gap theory and perception of value will add to the existing research. This is because this paper uses known theory and uses it in an area which has not been studied thoroughly yet, the BI systems market.
To be able to study the research question set forward in the last chapter this paper will include theory on BI systems, KPI's and perception gaps. Perception gaps are interpreted as likeness in perception in proposition 2 to get a better research model later. This theory chapter will have a discussion part when the theories are presented.
The strategic impact will be of importance through the chapter since the value of a BI system may be seen strategically. This in contrast to Carr's opinion in the article IT doesn't matter. A case study from India states the importance of the BI systems in the way that managers question the portfolio of information systems to gain the most strategic advantage (Gupta and Sanjay, 2004) . Jerry Luftman (2004) is one of many different authors who discusses the importance of the strategic alignments between IT and business. Another theory which is important to keep in mind is that IS projects has a tendency to fail (Sauer, 1993, Laudon & Laudon, 2000).
The strategic alignment is very important to discuss since there sometimes may appear mismatch between the KPI's and the business in daily operations. The KPI's in this case are measurements gathers by using the IT system. Hence, the outcome may not always be as useful for the managers as they initially wanted them to be. Measuring KPI's to achieve a goal for a company can be done in many different ways. For instance to measure customer satisfaction, KPI's could include order cancellation, late shipment, incomplete order shipment, returns and customer attrition (Wu, 2002).
Most of the time, BI simply means use of several Financial/Nonfinancial Metrics / KPI's to assess the present state of business and to prescribe course of action. The key aspects of this are presented below (Wikipedia, 2004). When BI usage do not indicate use of Financial/Nonfinancial Metrics / KPI's this paper will refer to a term the authors of this paper calls simple measurement. This term refers to measurement which do not satisfy any of the key aspects presented below.
The first probable reference to BI is made in Sun Tzu's " Art of War " where he claims that to succeed in war; you should have full knowledge of your strengths and weaknesses and full knowledge of your enemy's strengths and weaknesses. Lack of either one might result in defeat (Wikipedia, 2004) . This illustrates how much businesses really have changed during time. BI is as you notice not a new subject, but automating and simplifying this process is the new part of it. Today the BI Systems are viewed as systems which gathers information from different sources to gain insight into strengths and weaknesses of competitors and the companies themselves. So the goal of BI is the same as in previous time, but the process has changed with the use of BI systems. Williams (2004) defines BI Systems as systems which help management analyze a company and can create strategic business value.
According to Jonathan Wu (2004) there is a trend toward standardizing on a couple of BI tools. This trend contributes to market consolidation. BI vendors are trying to pack as much functionality into their product suites as possible in hopes of being chosen as the standard. Another article by Rick Sherman (2004) says that the BI trend with most press and marketing hype is performance management (BPM). This system is, as offered by the vendors, a combination of BI tools, pre built analytics supporting metrics or KPI's. The main advantage of this is that more time could be spent on the data analysis and less time gathering the data.
Williams (2004) states the importance of using KPI's in the evaluation of a company as this indicates the health of the company. KPI's are quantifiable measurements, which should reflect the critical success factors of an organization. These measurements will naturally differ depending on the organization. An organization can gather a lot of data, but it is the metrics that transform it into meaningful information that are important. That meaningful information can be a KPI for the firm. An example of an KPI used to measure customer satisfaction can be the percentage of the income which comes from returning customers (Reh, 2004) . A high percentage of returning customers can indicate customer satisfaction based on the thinking that an unsatisfied customer would not return.
Whatever KPI's are selected they have to reflect the organization's goals, be key to its success, and be quantifiable. KPI's usually are long-term considerations. The definition of what they are and how they are measured do not change often. The goals for a particular KPI may change as the organizations goals change, or as it get closer to achieving a goal (Carman and Conrad, 2000) . Another way of putting this is that the KPI's should be defined by keeping the goals, objectives and strategy of the company in mind, hence the KPI should directly be traceable to the overall strategy of the company (Griffin, 2004) . However it is important to keep in mind that KPI's should not create reluctance for special projects that will meet the strategic goals in the long run. To hinder this it is important to not putting too much emphasis on performance according to specific numbers.
When purchasing a BI system the company is facing a dilemma of choosing about 20 KPI's from around 100 metrics, which normally are included in the system. In addition there are several important issues to keep in mind. One of them is to ensure data availability to support the metrics (Bauer, 2004b) . As seen in figure 1 (Bauer, 2004b) defining the KPI's are dependent on several steps to be efficient in use; creating strategies, objectives and critical success factors.
Figure 1 – Bauer (2004b) - Strategic Alignment Pyramid
The strategic alignment is very important to discuss since there sometimes may appear mismatch between the KPI's and the business in daily operations. The KPI's in this case are measurements gathers by using the IT system. Hence, the outcome may not always be as useful for the managers as they initially wanted them to be. Measuring KPI's to achieve a goal for a company can be done in many different ways. For instance to measure customer satisfaction, KPI's could include order cancellation, late shipment, incomplete order shipment, returns and customer attrition (Wu, 2002) .
The implantation of BI systems in a company can create perception gaps between different stakeholders in the organization. “A continuously widening gap will cause business to lose its legitimacy and will threaten its survival (Sethi, 1979) .” This sentence gives an indication of how too high expectations can be crucial for an organization. Reichart (2003) states that underlying mental models held by different stakeholders selects different facts as being relevant to the issue at hand. These differences in relevant facts can create expectational gaps.
Work by Wartick and Mahon (1994) and Wartick and Wood (1998) present three types of gaps that can emerge: Factual gap, Conformance gap and Ideals gap. Each of these types of gaps calls for different response. The factual gap involves arguments between two parts regarding inconsistencies in the facts (“what is” versus “what is”). Factual gaps need objective responses to clarify in a matter that does not create misunderstandings. The conformance gap take the concept of “what is” and put it up against “what ought to be”. The perceptions of one party about inconsistencies in how another party should behave in light of their current behavior. Since none of the parties are generally willing to give up their own interests, especially when no one else has to lose, negotiated responses are more likely to be effective. The third type gap is Ideals gap which entails inconsistencies between two parties perception about what one party ought to do (“what ought to be” versus “what ought to be”). This ideal gap calls for a dialog and debate between parties regarding the values and beliefs involved since facts are not in question.
Gartner Group's so-called hype curve (figure 2) has to purpose of describing expectations, initiatives and penetration of e-technologies (Dahlberg and Hørlück, 2001). It might happen that the users are very satisfied with the system in the beginning, since they are excited for a new system that simplifies their work procedures significantly. However, after some time of use they might realize that the system is not as good as it should be and the value perception of the system decreases to a level that reflects the reality better.
Figure 2 – Gartner Groups hype curve with expectations and time
The benefits from an IS project may occur in different stages of the project according to the IS Innovation Phases (Larsen and McGuire, 1998) . These stages are Idea Phase, Creation Phase and Usage Phase. The focus within this paper (or web page) is to look at the usage phase. In the usage phase the change anchoring, change refinement and change termination may be found. The reason for choosing the usage phase is that a lot of the changes occur when the system is used. The focus in this paper is to look at how the number of KPI's change during use of a system. The thought is that users will realize new needs and request new KPI's to be used.
To be more specific this paper will look at the change anchoring and change refinement of the usage phase. The change anchoring phase is freezing IS functionality when effects are recognized. In a BI system situations, where a planned effect (for example: availability of KPI's) can be utilized, can be identified. The change refinement parts of this phase use the recognized effects to initiate new projects. In this paper the business executives may see a potential for different KPI's and initiate new projects to utilize these. The goal with the topic chosen is to look at how unplanned effects can occur and be recognized as an important effect as just given an example of. This will provide a basis for discussing the evolvement of effects over time.
Perceiving the intended role of a BI system from a vendor point of view and the user manager point of view may identify a gap of perception. The consultants could, to a certain degree, be responsible for this mismatch in the authors opinion. This mismatch can be the results of either lack of insight or lack of experience with a specific firm to be able to implement the systems as they should be (Barberg, 2004) . Another thought may be that the vendor hypes the system so the managers who are to use the system may have unrealistic expectations. The managers are not totally without guilt in this process as they should be critical to the process and not believe that the world is easy and wonderful, but not too critical as then the implementation would not succeed. It might happen that the users are very satisfied with the system in the beginning, since they got a new system that simplifies their work procedures significantly. However, as mentioned earlier, after some time of use they might realize that the system is not as good as it should be and the value perception of the system decrease. If the customer interprets that he has got what he asked for there is no reason for him not to use the system. This also affects all the users that have access to the information in the system.
The customer will be satisfied with the value if he got more value from the system compared to the costs. If the value is less than the cost, the project should be reviewed critically as this may indicate a failure. In the theory part different types of perceptual gaps (Reichart, 2003) were mentioned. In this case this gap is about value of the BI system from the vendors point of view and the customers/users point of view. This situation could be described as an ideal gap. If the customer can realize that the BI system gives them as much value as the vendor perceive, they probably will be more satisfied with the system and use it more. The gap between the perceived value of BI systems for a manager and a vendor is also important when looking at the level of KPI support and the number of KPI's. A vendor may perceive that a company is at a higher level then the manager of the same company realizes. As the system has been in use for some time the authors of this paper believe that the gap should be less then at the pre-usage phase.
The size of BI systems projects can be different from organization to organization. Some systems are used only on operational level or just strategic level, while other systems are used in both. Number of users indicates of what size the project is. Systems concerning operational functions will probably have many more users than systems used on strategic level only. Users can also have different type of authorization for the system. One example can be that some users in an economic department have access to employee's salaries, while other users cannot .
The trends discussed by Wu (2004) and Sherman (2004) will probably lead to an increase in the numbers of KPI's integrated into the BI systems. This approach can be a benefit for users who might get more value for the money. The value gained will then be in number of predefined KPI's, which only needs to be configured to the organization. One may also speculate if more predefined KPI's may lead to too many KPI's in the organization. A manager should be able to do more than just look at KPI's all day. It is difficult to say exactly how many KPI's should be used for measuring the performance in achieving a goal in a best possible way. It might be that some BI systems provide improved quality of the current measurements to compensate for other (new) KPI measurements. This could result in less KPI's as the managers use the BI system. However, the BI systems may have the capacity to measure a lot of KPI's. Hence, it makes sense to define as much KPI's as possible given that they are aligned with the business objectives.
The BI systems gives room to create and handle many new KPI's which have a lot of value for the company in sense of giving them the possibility to measure how well their organization is doing in different areas. However, to get fully advantage of the BI systems, the KPI's must be well defined and be neatly defined and implemented into the systems having in mind what kind of effect the organization want out of the indicators. It was mentioned earlier that managers from India question the portfolio of information systems to gain the most strategic advantage (Gupta and Sanjay, 2004) . In an analogy it can be argued that managers might request new KPI's after some time for the same reason.
A BI system may have different be support for KPI's. This support is best described as the availability to create and develop indicators tailored to the company need. The level of IS system is also crucial in the matter of support for KPI's. A large extent of information system support may indicate a large amount of data gathered which can be used in developing KPI's.
The business manager's use of KPI's is to be measured by the number of KPI's used in the daily operation of a company. It is also important to look at how widely integrated the BI system is in the company. This can be measured by studying how many of the total amount of managers who have access to the BI system.
Model 3 - The Research Model
One of the most radical choices one has to make in designing a research is between quantitative and qualitative approach. The differences between qualitative and quantitative methods are related to whether the data can be quantified or not. Decisions about which kind of research method to use may be based on the researchers' own experience and preference, the population being researched, the proposed audience for findings, time, money, and other resources available (Hathaway, 1995) . In this case the population was quite small and the time available to spend on the qualitative approach was limited. In addition the theoretical discussion and the research problems described above comprise the need for data that are difficult to quantify. Hence a qualitative research method appears to achieve the purpose of the research problems described above.
Qualitative data analysis is subject to major criticism from more ”quantitative” researchers due to lack of strict guidelines, and therefore might admit personal bias in interpreting the data (Palmquist et al., 2004) .In addition, according to Eisenhardt (1989) , there are as many ways to analyse qualitative data, as there are researchers. This might create problems, but it also allows investigating the data from several different angles that might discover significant findings.
The unit of analysis chosen for this project is users of BI systems and vendors of such systems. The unit of analysis is twofold since the sample collected are two different units. The next step is to investigate the gap of perception between a vendor of a BI system (what was promised from the vendor) and what a user of the BI system perceive as the benefits during usage. To analyze during usage the user will be asked to try to remember what his perception as the system was first introduced was and what his perception of it is today. This will capture the changing effects over time. The sample in this project is a consultant who sells BI systems and a manager who uses this system. Even though the user interviewed preferably will be the customer which the vendor are questioned about this is not a necessity. The customer interviewed can be another customer then the vendor talks about in the interview but this affects the validity and reliability of the study.
The protocol analysis went without major problems and most questions were agreed upon without any discussion (Appendix 2). After discussing the ones the authors were disagreeing on, an agreement was accomplished. The referee function was not needed this time. This could be a result of strict guidelines, mutual understanding of the questions, few interviews and clear answers to the questions.
Hammersley (1992) defines reliability as “the degree of consistency with which instances are assigned to the same category by different observer on different occasion “. Hence, it could be asked: “Would the same results be achieved in case someone else conducted this study?” Validity is explaining the degree to which a measurement actually measure or detects what it is supposed to measure.
In this case the purpose was to interview two persons from the authors choosing. The authors chose to get in touch with a vendor of BI systems and a manager using BI systems. To gain as much reliability as possible there was a need to interview the vendor asking questions regarding a customer that have implemented a system. On the other hand a manager that has some experience with use of a BI system should be interviewed. If the customer questioned is not the same customer that the vendor is questioned about, the reliability is higher due to that the “occasion” should be different. However, this could decrease the validity of the research, since the validity would be high when the same user as the vendor is questioned about is interviewed. However this paper had to increase the reliability because the customer which the vendor was asked about did not have time for an interview. The increase of reliability gives a broader use of this research and other researchers do not have to do the research all over again to use this paper.
See appendix 1 for interview guide (linked)
See appendix 2 for interview protocol and protocol analysis (linked)
Interview transcripts will not be linked due to privacy issues and will be, if needed, presented for grading at request from the lecturer.
The results presented in this analysis are a result of a coding of the interviews and are based partly on the author's perception of the answers. This was done to be able to analyse the results in a systematic manner. The analysis is presented in the way that the results are displayed in a table and the results are discussed below the table.
The table heading states the R-1 and R-2 are the
respondents. R-1 is the vendor and R-2 is a customer using the vendor's
BI system. Further information about the interviews can be obtained
from appendix 2, which contains the interview protocol and the protocol
analysis.
|
|
|
R - 1 |
R - 2 |
|
|
|
|
|
|
1.1 |
AGE |
39 |
29 |
|
1.2 |
EDU |
Medium |
High |
|
1.3 |
MANRESP |
Low |
Medium |
|
1.4 |
PLACE |
Low |
High |
|
1.5 |
POS |
Consultant |
Controller |
|
1.6 |
EXP |
High |
High |
Table 1 - Results background and initiation
The first table presented contains mainly background information and demographic information about the interviewed people. In both cases they have a lot of experience, but only the customers have an education that satisfies the high attribute. None of the respondents have management responsibilities and the reason for putting medium in the customer case (R-2) is that the interviewed customer representative is at a higher level in the organization (a support function). The interviewed vendor was a consultant and was at the bottom of the organization. This is also represented in the item place that states that R-1 is low in the organization and that R-2 is high in the organization. Both of the respondents have worked with projects several times before and this qualified for the high attribute in both cases.
|
|
|
R - 1 |
R - 2 |
|
|
|
|
|
|
2.1 |
INTFUNC |
High |
High |
|
2.2 |
PRESPER |
High |
Medium |
|
2.3 |
NODEL |
Medium |
Low |
|
2.4 |
DIDDEL |
High |
High |
|
2.5 |
INTFUNCOP |
High |
Medium |
|
2.6 |
PERSYS |
Medium |
Low |
Table 2 - Results last implementation of BI software and the promised functionality
Table 2 is presenting the results for questions asked about the last implementation of BI software and promised functionality. The vendor was of the opinion that it was promised a system to the customer with extensive functionality. The customer interprets the promised functionality in the same way. The present perception of the BI system shows that there is a difference in perception between the vendor and the customer interviewed. The vendor perceives the functionality of a BI system as extensive and good; while the customer realizes that the system they use has some limitations and grades the functionality of the system as average.
The BI system, which the vendor is talking about, had not implemented some functionality that can be characterized as non-critical. However, according to the vendor this was due to the data quality of the customer was bad and created a problem of using these data. Given that, the vendor is very satisfied with the system delivered to the customer and is of the opinion that all the most important functionalities was delivered. The other respondent was satisfied with the functionalities of the BI system. Even though it has its limitations the customer says that they got all the functionalities they wanted when implementing the system. The vendor believes that the intended functionality and scope of the system from their customer's point of view is that the system is extensive. While the customer interviewed understood that the vendor look at their system implementation as a medium sized project.
There were observed some changes in the perception
of the system during the implementation and use. The vendor observed
that the attitude towards the BI system was less positive after some
time. This might have been a result of that the users realized that the
new system simplified a lot of their daily operations, but they
realized that they wanted more functionality.
|
|
|
R - 1 |
R - 2 |
|
|
|
|
|
|
3.1 |
KPIOPER |
Medium |
Low |
|
3.2 |
EVALBUIS |
Medium |
Low |
|
3.3 |
INCKPI |
Medium |
Low |
|
3.4 |
IDENTKPI |
Medium |
Low |
|
3.5 |
ACCKPI |
High |
- |
Table 3 - Results KPI
Table 3 is presenting the results for the questions asked about KPI's. The vendor had observed that their customers did not emphasize on the use of KPI. However, there are few KPI's that are operationalized by using BI systems. The vendor also mentioned that the reason for this is often that the accessible data is of poor quality. One example could be inventory management. The customer interviewed did not use any KPI's at all. However, they used simple measurements gathered from different sources.
The vendor was not sure if the customer did use any KPI's to evaluate the business before implementing the BI system. All the information around what kind of KPI's that were supposed to be implemented was delivered by the customer themselves. However, the vendor assumed that they used KPI's in some degree before implementing the BI system. The other respondent was using simple economics measurements even before implementing the BI system. The vendor knows that a few new KPI's has been implemented into the BI system they delivered at a later point in time. This is due to better data quality that allows for these new KPI's. The interviewed customer has not yet begun to use KPI's.
The customer who the vendor was asked about is
using the BI system on the operational level. This is where most of the
employees have access to the logistics functionality of the system and
was also the main intention of the project. However within the economic
part there are some access restrictions. The interviewed customer does
not use any KPIs, but when it comes to the measurements used, a lot of
users on tactical level have access.
|
|
|
R - 1 |
R - 2 |
|
|
|
|
|
|
4.1 |
OUTSUC |
High |
High |
|
4.2 |
OUTFAIL |
Low |
Low |
|
4.3 |
KPICNTBTE |
Low |
Low |
Table 4 - Results Outcomes
Both the vendor and the customer interviewed
regarded the outcome of the BI system implementation as successful.
However, the vendor mentioned that there was a small degree of failure
since some functionality could not be implemented. Neither of the
respondents sees any contribution to the outcome of the BI system from
the KPI's involved.
|
|
|
R - 1 |
R - 2 |
|
|
|
|
|
|
5.1 |
PERFUNCCAP |
High |
High |
|
5.2 |
PREVBIPROJ |
Low |
Low |
|
5.3 |
VALUECOST |
High |
High |
Table 5 - Results about BI systems in general
Both the vendor and the customer have extensively and realistic perception of the capability of the BI systems in general. The consultant has been involved with a lot of BI projects and have experienced that KPI's do not contribute to the outcome on a general basis. The customer has not been involved in any BI projects earlier. According to the vendor the value of money is dependent on the project cost, but was sure that the BI systems gives more value to the customers than the amount of money spent on them. The customer interviewed was sure that the BI system has given them more value than they have paid for it. What is important to notice here is that the outcome was successful and very low failure rate in the systems.
Our research question for this chapter was: “ What are the relationships between the perceived value of BI systems, the likeness in perception between vendor and customer and the business manager's use of KPI's?” This section will discuss the results presented in the analysis and the reason for the outcomes.
The first point to notice in the analysis is the project success rate. As seen from the results, BI projects tend to be successful and have very low failure rate. Both the vendor and the customer were sure that a BI system gives more value than it cost to acquire it. This is in opposition to Sauer (1993) and Laudon & Laudon (2000) where they state that IS project tend to fail. Perhaps this could be an indication that BI projects are IS projects with a low failure rate or that the IS industry has matured and do not experience as much failure as earlier. This result does not influence the result of the propositions, but could be a future research.
Does increasing perceived value of the BI system increase the business manager’s use of KPI’s?
Both customers knew about the value of BI and want to use the system for more than it is used for today, but due to bad data quality and the fact that the system was acquired without having use of KPI's in mind they are not able to use KPI for measuring the companies' performance / health. Hence there is no indication that the increased perceived value of the BI system increases the business manager's use of KPI's. This is in opposition to what the trends, indicated by Sherman (2004) and Wu (2004), said. The analysis of the data collected can also be an indication that managers do not to a large extent question the IS portfolio as indicated by Gupta and Sanjay (2004).
|
Increasing perceived value of the BI system increases the business manager's use of KPI's |
Not Supported |
Table 6 – Outcome of propositions 1
Overall this leads to a rejection of the first proposition as the data collected did support the proposition, but it may not necessarily be so since a larger data sample could end up with a different result.
Does an increasing likeness in perception of value of the BI system between vendor and customer increase the business manager’s use of KPI’s?
The results showed that there are almost no
perceptual gap between the vendor and the customer when it comes to the
value of the BI system. As mentioned there occurred a minor gap after
some time as perceived by the vendor. This can be pictured by the hype
curve (Dahlberg and Hørlück, 2001). The new system had a
lack of
ability to reserve items and that was probably perceived as more
important after some time. Also this can be referred to Gupta and
Sanjay (2004) where they talk about how management questions the IS
portfolio during usage. The questioning of the IS portfolio indicated
by Gupta and Sanjay (2004) is the same as in the first proposition
where it was said that the managers do not to a large extent question
the IS portfolio, but here we might add that some questioning is made
(not much). This can be the reason why the management has been slightly
less positive about the system as the reason for the lack of this
functionality is somewhat bad data quality. This perceptual gap can be
classified as a conformance gap where the concept of “what is” is put
up against “what ought to be” (Wartick and Mahon, 1994). However, the
customer we interviewed has not had any changes in the perception of
the system since it was implemented. As mentioned earlier they were
satisfied with getting what was asked for and what was promised. It
might have happened that if the data quality was better, the customer
the vendor talks about would have taken KPI's in use. The other
customer mentioned that the systems capacity is limited when using it
for more complex tasks. If the system was intended to use KPI's it
might have happened that the customer would have taken those into use,
and even increased the use over time.
|
An increasing likeness in perception of value of the BI system between vendor and customer increases the business manager's use of KPI's |
Not Supported |
Table 7 – Outcome of propositions 2
The results here indicated that one company did not perceive any gap and another one perceived a small gap. Whether this gap leads to more or less use of KPI's is not indicated. As the increased use of KPI is not supported the proposition is rejected, but it is not necessarily so because a larger sample may yield a different result. Still based on the data gathered the proposition is rejected.
Overall there are some interesting findings from the interviews, and it is possible to assume a lot around the circumstances, but the results do not support any of the propositions presented in this chapter. The reality is not necessarily so since the data sample is small so the outcome is not based on a valid sample. The research has low validity but indicate several important aspects discussed above.
As the data was analysed some ideas for refinement were developed. These ideas are presented and explained in this section. The first one is regarding the perceived value gap. Increased gap may lead to an effort of improving the KPI's and hence adding more KPI's to the system. This is a very interesting aspect since we initially perceived that the higher likeness in perception increases the KPI use.
Will increased gap of value perception lead to an increase in the use of KPI's?
This new proposition is in relation to proposition 2, but is concerning wheter the opposite of proposition 2 is true. That is if a large gap leads to a higher increase in perception of value over time and not the increased likeness in perception of value. The difference here is that if proposition 2 is rejected, then it would not be obvious that this new proposition would be true. That is why we suggest this as a refinement of the research model.
Number of KPI's can be a little misleading, since it might happen that use of BI can make a lesser number of KPI's more efficient than without using BI. The paper is limit to a specific department within a company because of the amount of work to be used in this study. Also the study of the perception gap is difficult since it involves just a few variables and mainly the variable if the vendor and the customer perceive that there is a gap. A larger study would have used a lot more specifically defined variables to study this concept.
Studying the business manager may not be the correct term to use when talking about BI systems. Users of such systems may be in the support unit of the organization and these do not have managerial responsibility. These users are valid for this study, but do not technically fall under the term business managers.
Probably the most important limitations to this paper is the time frame of its development as it limits the time to get hold of just a few interviews. These interviews are very limited in number and that is why the results in this paper is not as important as the methodology used to get to the results. Two interviews as conducted here are not highly significantly valid.
As mentioned in the introduction the purpose of this research was to put focus on the use of KPI's in the BI system. The BI system which was studied did not indicate that KPI's are used in organizations and that media has focused too much on this term. The study did uncover simpler measurements then the KPI and this should be studied further in a future research.
None of the proposition were supported since the use of KPI was very little or none at all. This results in a study that needs more data to investigate the relationships. The current study lacks data on the dependent variable, but useful information was obtained in the study anyway, which indicates future research in this area. For the research question this means that a relationship between the perceived value of BI systems, the likeness in perception between vendor and customer and the business manager’s use of KPI’s cannot be confirmed. This does not mean that it is rejected either since there is little data gathered. The conclusion of this paper is that there supposibly is a relationship between the constructs, but they were not revealed in this limited study.
The proposed model did not get any support from the data collected. The use of KPI can still contribute to the perceived value, but the interviews did not indicate a use of KPI's to a sufficient degree. The best indicator of this is that the consultant had never been involved with projects where the KPI contributed to the outcome of the project. So the dependent variable did not turn out to be valid. The likeness in perception of value was observed as high which is very good. A high value tells that both the vendor and the customer perceive the BI systems capability to an equal degree. That is to say that there is low degree of overselling from the vendor. What was the most interesting finding was the use of more common measurements, which are implemented to let the users get faster access and collect data from different sources. Very few combine these data from different sources so that new value is gained.
The implications for practice of the results of this paper may indicate that use of KPI in BI systems may be overhyped. Perhaps the knowledge of goals and visions are not as extensive as needed to be able to use KPI's in the corporate setting.
For research this paper creates a good theoretical foundation for future research. The findings in the paper is limited by the time and the number of responedents asked. Since this study only used one BI system these results may not apply to other BI systems.
Future research could be to test these results in an empirical setting. Another research proposal could be to investigate the level of BI usage and what functionality that actually is used in the BI system. Media is always talking about trends that cannot be observed in the real market. Whether these trends actually do happen and how long it takes before they happen is another interesting research proposal. A third future research could be to study what this paper called the simple measurement and perhaps discuss if this is a step in the process of implementing KPI's. A fourth future research is to look into wheter BI projects are more successful then IS projects in general.
Appendix 2 - Interview protocol and protocol analysis
This chapter of the book is examine if the
perceptual gap (differences in perceptions), in the early stages of an
IS innovation, between IT/IS experts and future end-users of an IT
system changes. The innovation is the introduction of an Enterprise
Content Management (ECM) into an IT consultancy company.
The research is done by developing a scenario, introduces it to the future end-users, and explores if the use of a scenario will alter perceptions held by the IT/IS experts and the future end-user of the system. The research expect to find that the IT/IS experts’ perceptions of what the systems should be and the future end-users’ perception would move closer together. The results are not known yet, but will be included when the research project is finalized.
As information technology becomes more inherit in
the modern company functions and processes it is becoming a
prerequisite of doing business. Experience has shown that in today’s
world an implementation of a new IT system can make or break a company.
That coupled with the fact that up to 80% of IS development efforts
fail at least partially ((Sauer, 1998), Vowler, (1991) and Mowshowitz
(1976) in Larsen (1998)) illustrates the need for finding ways to
improve the development process and thus increase the likelihood of
succeeding.
Research of innovations has often been focused on explaining the
diffusion of already-developed innovations (Rogers, 1995). The focus
has frequently been on the development process, (Avison and Fitzgerald,
1995, Davenport, 1993, Larsen, 1998, Van de Ven, Polley, Garud and
Venkatraman, 1999) the end-user evaluation in late stages of the
process (Davis, 2004, Davis, 1989) and after it has been implemented
(Doll and Torkzadeh, 1988, Goodhue, 1995). Relatively little is known
about generation of novel ideas and early phases of and innovation
process (Van de Ven et al., 1999). The focus of this chapter will be on
these early phases of the development process, or more specifically the
Idea Creation Phase (Larsen, 1998) also called The Fuzzy Front End
(Cagan and Vogel, 2002). It will look at if and how the use of a
scenario helps to bridge the gap of perceived benefits of the IT system
between the IT/IS Expert and the end-user.
Scenarios is found to be a good way of describing
information systems design at an early stage (Schaik, 1999) and there
is a general understanding that it is beneficial to involve end-users
early in the design of an IT system (Hunton and Beeler, 1997, Schaik,
1999). However, innovations are surrounded by much uncertainty about
perceived benefits compared to existing systems (Rogers, 1995).
Describing scenarios as “vivid description of plausible futures”
(Lindgren and Bandhold, 2003) it should be useful in describing the
effects of a system. In this respect the authors will present the
end-users with a scenario in order to evaluate the effects of a future
IT system. Their perceptions can then be measured and compared with the
IS/IT expert’s perceptions. This exactly what will be done in this
chapter of the web book, it will look at the IS/IT experts perception
of the system effects and see if it changes during the scenario
creation process and compare the results with the end-user perception.
In order to this the authors will use action
research executed in by two stages. Stage one is pre scenario creation.
At this stage the authors will measure their perception of an
Enterprise Content Management system (ECM). The idea will then be
developed into a “conceptual prototype” by using a scenario. As a stage
two the authors will measure their perception of the ECM system to see
if the process of creating the scenario has changed their perception.
Finally future end-users perception of the ECM system will be measured
to identify the difference between the IS/IT experts and the end-users.
This chapter of the web book will firstly deal with the theoretical foundation of the chosen topic. The topics covered are innovation, innovation and uncertainty, scenarios, how users should be involved early in the innovation process and measurements of the benefits of scenario use, early in the innovation phase. The measurements discussion leads to the rationale for the chosen research question, and establishing the two research propositions. It continues by introducing the measurements for the detailed research mode, the research model itself, coded for an explanation of the corresponding detailed measurements, which is then outlined in a table and an explanation of the research methodology. Furthermore, this version of the chapter continues by analysing the results of the interviews, in accordance to the research question and a discussion of the results. Moreover, the chapter contains a separate part were other interesting aspects or findings are included and discussed. Then the chapter covers theoretical and practical conclusions before reflecting on the project in it self.
In this section the theoretical foundation for the
chosen topic is explained by discussing how scenarios can be used in
the early phases of IS innovation processes, by discussing innovation,
innovation uncertainty, the importance of involving users at an early
stage of the IS/IT innovation process and explanation of why the use of
scenarios are found to be interesting.
In order to achieve this chapter will firstly discuss how innovations is believed to be following a series of stages, in which the idea needs organizational support, otherwise it will not make it trough to the next stage. Secondly it will move on to discussing reasons reason for why an idea is not taken to the next stage which is believed to be the uncertainties of innovations. Then it explains why scenarios are found useful in order to reduce these uncertainties and to understand and explain the needed perceived social recognized probable effects of an innovation, thus securing the needed organizational support to move an innovation trough the stages. The above theories acts as the rationale behind the research question, propositions and corresponding measurements of the benefits of scenario use in the early stages of the IS/IT innovation process and concludes by introducing the research model.
Management and scholars are most often assuming
that the emergence of novel ideas is following a process, trough a
series of stages (Van de Ven et al., 1999). However, the uncertain
nature of innovation processes has led scholars and managers to also
view the innovation process as a random process (Van de Ven et al.,
1999). Both are valid, but due the nature of the research being carried
out and because of how the authors are aware of the recommendations of
other scholars (Cagan and Vogel, 2002, Larsen, 1998) the process for
the idea is seen to follow a series of stages.
The stage model chosen is Larsen’s (1998) IS Innovation framework. As the idea is an IS innovation to the company, Larsen’s framework fits well. The idea, a bottom up Enterprise Content Management System (ECM), is at the very first stage of this framework, the idea creation phase. An idea, at any stage, needs organizational support to develop trough to the next stages (Larsen, 1998). An idea which does not get the support from a decision maker will not make it into the idea creation phase (Larsen, 1998). One way of getting the needed support is reducing uncertainty and show potential benefits for the organization and its users.
Rogers (1995) argue that innovations, in general, create uncertainty. Uncertainty implies lack of predictability, structure and information. According to Rogers (1995) technology often reduces the uncertainty of an innovation, based on the cause-effect relationship on which the technology is based. Innovations often have some degree of benefits for the users. They are often not so easy to see, and users might seldom be certain on how, and if, the innovation represents a alternative to what they have done or used before (Rogers, 1995). Rogers further argues that innovations creates uncertainty of the expected consequences, but reduces uncertainty by the information base embedded in the innovation itself. This is probably true for an innovation, or technology which the end-user can touch and feel. Products which is possible to touch and feel, is by definition, in the usage phase of the IS innovation framework (Larsen, 1998). Based on the above, the authors’ innovation can create much uncertainty and it is probably even more difficult to see the benefits of the innovation this early in the phase, than in the usage phase. Thus, it is difficult to get the acceptance of the innovation, which is needed to take the innovation into the creation phase (Larsen, 1998). According to Ahn and Skudlark (2002) scenarios, and scenario planning, can be used to gain insight into a new service’s viability and thus reduce uncertainty.
Our brain is using scenario planning everyday. “The
healthy brain is constantly “writing up” scenarios, interpreting
signals in the environment and reframing them as meaningful images of
and trajectories of the future. Healthy organizations do this too”
(Lindgren and Bandhold, 2003 p1). There are many definitions of
scenarios and scenario planning: “ a internally consistent view of what
the future might turn out to be”(Porter, 1985) in Lindgren and Bandhold
(2003 p21) , “a tool [for] ordering one’s perceptions about alternative
future environments in which one’s decision might be played out right”
(Schwartz, 1991) in in Lindgren and Bandhold (2003 p21) or “a
disciplined method for imaging possible futures in which organizational
decisions may be played out” (Shoemaker, 1995) in Lindgren and Bandhold
(2003 p21). This chapter is not trying to adopt any one of the above
definitions specifically; they are included to give the reader a quick
idea of what it can be but the following quote summarizes rather well
how the scenario can be used to investigate the end-users perceived
benefits of a future system: “A scenario is an answer to the question
of what can conceivably happen, or what would happen if …” (Lindgren
and Bandhold, 2003 p21)
According to Lindgren and Bandhold (2003) a good scenario is made up of
seven criteria. Four of these criteria do not apply to this research,
as it is concerned with the development of more than one scenario. The
first of the criteria applicable to this research is how it should have
decision making power. This means to make sure it gives enough insight
into the question analyzed. Secondly, it needs to be plausible, or fall
into limits of what is realistically possible. Thirdly there the
scenario should challenge the individuals or the organizations wisdom
of the future. These three requirements are prerequisites of a
successes full use of a scenario and will therefore be used to measure
if the scenario is being used correctly and efficiently, end thus
meeting the criteria of scenario use. However these are not the only
measurements possible, there are others like claims analysis (Carroll
(1992) in Schaik (1999)), but the above ones were found to be best
fitted to this research.
Since scenarios are useful in describing the functionality of a system early in the design process, and can be used as prototypes of sort (Schaik, 1999). This research will investigate if scenarios can be used to improve the early stages of the IS/IT innovation process by using a scenario.
There is a general understanding that it is
beneficial to involve end-users early in the design of an IT system
(Hunton and Beeler, 1997, Schaik, 1999) Schaik further argues that
end-users can be involved in task analysis (Johnson, 1988), and testing
of prototypes (Nielsen, 1993) and how this is often seen as a continuum
from very technically centered (least end-user involvement) via joint
user-specialist involvement to user-led (most end-user involvement). On
the other hand, even though it is desirable to involve all end-users
and as many specialists as possible, there is a limit to what is
feasible. Thus, in many IT design projects only include specific
end-users, selected for the task, based on some rationale criteria,
such as super users, most involved users, managers etc.
Early involvements of end-users have many
advantages. Damodaran (1996) mentions five benefits: improved quality,
avoiding costly end-user features the end-user will not use, improved
levels of acceptance of the system, greater understanding and more
effective use, increased participation in the decision making in the
organization. She further argues that these benefits are not gained if
the end-users are not involved early on in the stages of planning and
design. Hunton and Beeler (1997) found three main implications of
involving end-users in the early stages of an IS project. The first
implication is involvement, leads to higher actual participation in the
process. Thus those end-users which find IS personally relevant should
be given the chance to participate. Secondly, even though end-user
involvement is high, self-efficacy is low and unless end-users
participate in the process, the self-efficacy perception is unlikely to
change. However, last but not least, involving and getting end-users to
participate in the IS development process one maximize the end-users
instrumental control over the system, thus most likely gets the system
closer to end-users needs and wants.
This brings on an interesting focus on the
end-users needs and how the IT/IS expert perceives them. According to
Cagan and Vogel (2002) perception comes from sources like, education
and inherent personality of a profession. They argue that these
differences will lead to perceptual gaps. “Perceptual gaps are the
differences in perspectives that team members have that stem from
discipline specific thinking” (Cagan and Vogel, 2002). In the IS/IT
context this means that IS/IT experts and the end-users will have
different perspectives of an IS/IT system in development. This
difference can be disruptive in the development process and lead to
conflicts (Cagan and Vogel, 2002). Therefore it is highly important for
the success of the development process to address these different
perspectives.
Scenarios are used as prototypes, or specifications of functionality, early in the development process of a future system as a mean to help end-users perceive future benefits of a system (Schaik, 1999). Based on this it can be reasoned that the use of scenarios, i.e. introducing it to the end-user, will change his perception. In addition scenarios are commonly created by the IS/IT Experts as a tool to specify functionality and benefits for the end-user (Schaik, 1999). From this the question arises; does the use of scenarios, i.e. the process of creating scenarios, change the IS/IT Expert perception of the end-users benefits. As the end-users and the IS/IT expert perceptions change, the perceptional gap between them also changes. A decrease in the perceptional gap is beneficial; therefore the benefits of scenario use can be measured by a decreased gap between the end-user and the IS/IT Expert. Hence the following research question:
Does increased use of scenarios reduce the perceptual gap between end-users and IS/IT Experts?
Measuring the perceptional gap is one way of
measuring the benefits of scenario use in the early stages of an IS
innovation process. There are a number of other ways of measuring the
perceptional gap like; predicting actual systems use (Schaik, 1999)
involving end-users at early stages of the development process (Hunton
and Beeler, 1997) idea acceptance (Knol and Stroeken, 2001) and
managing risks (Ahn and Skudlark, 2002). The reason for researching the
perceptional gap is that there is not found to be any published work in
an IS/IT context. However, it is found to be an important issue in
product development literature (Cagan and Vogel, 2002).
The perceptual gap can be found in an array of different elements of an information system, ranging from screen layout, to cultural effects of the IS/IT system. In this chapter of the web book the view on the perceptual gap is taken from Larsen model (1999, p 861) “effect areas and their suggested causal relationships, with examples”. Still this framework is too large for the scope of this research therefore it is narrowed down to two aspects of that model: the individual actor effects, under the business “line” and the information needs and requirement effects. These were chosen because the authors have access to individual users, as interview objects, and the information needs and requirements were found highly relevant in previous research by the authors (Erlingsson and Grødem, 2004b). Therefore the research propositions:
Proposition 1:
Will IS/IT experts and future End-users of an IT system have more
similar perceptions of “individual actor effect” if a scenario is used
in the idea development phase of that IT system?
Proposition 2
Will IS/IT experts and future End-users of an IT system have more
similar perceptions of “information needs and requirement effects” if a
scenario is used in the idea development phase of that IT system?
In order to operationalize these propositions the
authors have found several studies that can be used to measure the
chosen parts of Larsen’s (1999) model. One if the main reasons for
choosing the variables used in this study was that they were found to
be highly relevant to the concerns and findings of a previous study
(Erlingsson and Grødem, 2004b) done within the observed company.
Another reason for choosing variables, and later measurements, is that
they should be as general as possible in order to fit different job
tasks within the company and be used in further studies, and. The
latter is especially relevant here, since this is research is limited
and needs a follow up study looking at the change in the end-user
perception. The final reason for choosing these variables was the fact
that most of the measurements found are focusing on post-implementation
measures and the fact that this study are measuring pre-implementation
perceptions, the choice of variables was limited.
In relation to the individual actor effects the research focuses on the perceived impact on task productivity and task innovation, introduced by Torkzadeh and Doll (1999) and further developed by Davis (2004), and the perceived usefulness, introduced by Davis (1989). Within these researches work there are further measurements of individual actor effects and also other studies like Goodhue (1995). The measurements chosen, in addition to the above reasons, were found to be the most relevant measures of the end-user perceived effects in early stages of the IS Innovation process. In regards to the information needs and requirement effects investigated, this research focuses on, perceived ease of use, content quality and task- technology fit. These variables were all found to be important measures in previous research by the authors (Erlingsson and Grødem, 2004b) and thus of high relevance to the company the idea was developed in and in which this research carried out. These above measurements were also found to be generalizeable and supported in literature (Davis, 1989, Doll and Torkzadeh, 1988, Goodhue and Thompson, 1995).
The research model, Figur 1, in this chapter is
derived from the propositions and variables introduced in the previous
chapter. The research model assumes that scenario use will decrease the
perceptual gap between the end-users and the IS/IT experts in regards
to ‘Individual Actor Effects’ and ‘Information Needs and Requirements
Effects’. It does this by building on the previous reasoning about how
scenario use will help IS/IT experts and end-users see the end-users
individual actor effects and the information needs and requirement
effects of the system described and thus decrease the perceptual gap.
This research model (Figur 1) also shows the variables used as measurements for each construct. These variables, as discussed earlier, have been taken from various IS research and amongst other reasons chosen based on how they fit the company as found in previous research (Erlingsson and Grødem, 2004b) and the fact that they must be generalizeable enough to be used in a wide range of job task within the company and in further studies.
The methodology part of this web book chapter covers how the authors have used action research as the basis research design. Action research imposes challenges and has implications for a lot of issues, in which was experienced in its full right. The chapter cover the method chosen in order to write a scenario, and how the CPS method and a software program were found to be very useful. The scenario creation method is an adaptation of strategic scenario planning and writing methods. The longest section in the methodology is the data gathering section, which covers issues like how the authors have used semi structured interviews, combined with survey questions as the means of gathering data. This has both pros and cons, but in this research it was found to be valuable. Codification of interviews followed no very strict structure, but used the interview questions structure outlined above. The sample and population were taken from an organisation previously known to the authors, and therefore much of a convenience sample, suited to the research. The analysis method used is done in two steps, first, by looking at the scoring secondly, by analysing the results of the transcripts. At the end some limitations are being introduced. The most obvious limitation being how the end-users are not measured twice. This has great implications on the research.
Normally researchers are looking at a problem or
phenomena from outside of the phenomena itself. However, in this
research the researchers have chosen to involve themselves in the
process as the IT/IS experts, making this an action research. Although,
subjectivity issues never can be disregarded in any research this adds
additional epistemological and methodological challenges to the
research project (Day, 2002). These issues should not be disregarded or
overlooked. In order to counter some of the issues the authors have
taken some steps to ensure insight into the process. The concrete steps
taken at this moment are the writing of a diary, or a public weblog of
the process. This allows outsiders a peek into the process itself and
enables others to critique and comment on the research. This further
enhances author’s possibility of reflecting on the process for
additional learning. The blogs can be found at the following addresses:
http://www.ottarge.blogspot.com/; http://espeng.weblogg.no/bifall2004/.
Action research is found to be a very appropriate research method for
IS research (Baskerville and Wood-Harper, 1996) based on the fact that
IS is a very applied field, highly vocational and clinical in how the
researchers often have a helping role in the organisation (Baskerville
and Wood-Harper, 1996). The authors are conducting interviews as the
primary mean for gathering data and the research is in its final form
going to be presented, and hopefully acted on by the top management
i.e. the CIO.
With the above in mind the authors have chosen to apply only parts of the action research methodology as the framework for testing the use of a scenario as a means to get a better understanding of the perceived effects and generate more insight into how end-users perceive the proposed idea. Action research can be seen as “a flexible spiral process which allows action (change, improvement) and research (understanding, knowledge) to be achieved at the same time” (Dick, 2002). In action research researchers test a theory with practitioners (end-users in this case) gain feedback, and refine the theory as a result of the feedback and test again (Ahn and Skudlark, 2002, Baskerville and Wood-Harper, 1996). Iterations add to the research and with modifications and reflections the theory can be enhanced into something more useable. Even though the scenario and the system will benefit from iterations between end-users and researchers, this will not be the case in this chapter due to the time constraints of the interviewees and the class.
Scenario theory suggest (Lindgren and Bandhold,
2003, Ringland, 1997, Schaik, 1999) that creation of scenarios is a
comprehensive process of tracking, analysing, imaging deciding and
action upon information about social, economical, political,
technological, design or other trends which provide opportunities and
threats. This is a normally, in strategic scenario creation, a
comprehensive, continuous and time consuming process. This process was
found to be too time consuming, outside the scope of the course and
therefore not suitable for this research. However, by having conducted
previous research in the company the authors believed they have
sufficient insight and knowledge for this study. The outlined process
is therefore simplified to include information and knowledge held by
the authors. The above process started by using Creative Problem
Solving Method (CPS), as outlined in the book by Treffinger, Dorval and
Isaksen (2000), in combination with the MindManager Application as the
method and tool for brainstorming for ideas and features to be included
in the scenario. Using the CPS resulted in 80 ideas of features. All of
the 80 items were not features and the authors used MindManager to
structure the list into main features, benefits and criteria. The five
main categories were: Document search, quality insurance, easy to
monitor, networking and processes. The full list and a model of the
features, criteria and benefits list can be found as an appendix to the
web chapter. The authors used the main feature categories and their sub
features to write two independent scenarios. Thereafter these two were
combined into one. This was done in order to limit the possibility of
influencing each other.
The scenario was not tested on any users before conducting the interviews. The reason for this is that a test run of the scenario would be the first iterations to the scenario. This would influence the creation of the scenario and thus the measuring the effects of the scenario creation, which is the aim of the research.
The data gathering method is a semi-structured
interview combined with two short surveys. The reason for using both
interviews and surveys was mainly twofold. Firstly, the independent
construct “scenario use” was to be measured with as little influence
from the researches as possible and secondly the limited time the
interviewees had to participate. Furthermore, it reduces the need for
codifying the transcripts, having limited time at hand. Based on this
the interviewers decided to focus the interview on the dependent
constructs and have the discussion as open ended questions in order
gather as much data as possible. The surveys were then used to get
additional measures for each construct, both dependent and independent.
The structure of the interview, and the interview guide was thus this:
First the interviewees were asked about demographical questions which
could not be found on the company web site. Next they read the scenario
and answered the survey measuring the ‘scenario use’ construct. Thirdly
the interview was conducted, starting off by asking general open
questions in regards to perceived benefits of the system described in
the scenario and more concrete questions in relation to scoring if they
were not covered in the general open questions. Finally they answered a
survey asking specific questions measuring their perception in regards
to the two dependent constructs.
The variables, as previously stated, are grounded
in theory and the same literature has been used as a ground for
creating measurements and to create the interview guide and the
surveys. To create measurements, used in the interview and the surveys,
already tested questionnaires, researching the variables were used when
possible. For all but one of the dependent variables, the questions
were found in previously used questionnaires. Question Nq2, concerning
data quality, was not found in previous research but data quality was
frequently mentioned in previous research by the authors (Erlingsson
and Grødem, 2004b). The questions, taken from literature, had to
be adjusted due to the fact that they where intended for survey use,
but were, in this research also used as interview questions. In general
they were made more open ended. In the case of the independent variable
no previous studies were found. The questions regarding scenarios had
to be made from the variables introduced in Lindgren’s and Branhold’s
book (2003). All questions and literature references can be found in
Table 1.
In order to codify the interview answers the following steps have been taken: Each measurement has been coded with a number consisting of a capital letter indicating which construct it belongs to (S for scenario use, A for individual actor effects and N for information needs and requirements effects), small caps representing which variable it belongs to (d for decision making power, p for productivity, t for task-technology fit etc.) and finally a digit indicating measurement number. Thus Sd1 stands for question 1, on the decision making power variable under the scenario use construct. The next step taken was to create a rating scale both for the interview and the surveys. In the survey part of the interview a Likert scale of 1 to 5 (1 = strongly disagree and 5 = strongly agree) was used, while the survey like questions in the interviews got a rating from 2 to 4 (2 = disagree and 4 = agree). The reason for using two different scales is the difficulty determining the degree of agreeing or disagreeing in an interview setting. This latter rating was only used to indicate the over all general perception of the interviewee about each construct. The codification and the scales for each measurement can be found in Table 1.
Table 1 Statements used in the interviews
| Code | Statement | Measurement method | Scale | Reference |
| Sd1 | This scenario gives me good insights into this system | Survey | 1-5 | (Lindgren and Bandhold, 2003) |
| Sd2 | You understand how this system will work | Survey | 1-5 | (Lindgren and Bandhold, 2003) |
| Sp1 | It is possible to create this kind of system | Survey | 1-5 | (Lindgren and Bandhold, 2003) |
| Sp1 | This is a plausible future for the company | Survey | 1-5 | (Lindgren and Bandhold, 2003) |
| Sc1 | This scenario has changed the way you view the future for the company | Survey | 1-5 | (Lindgren and Bandhold, 2003) |
| Sc2 | Implementing this software will change the organization | Survey | 1-5 | (Lindgren and Bandhold, 2003) |
| Ap1 | Should it increase your task productivity? | Interview | 2-4 | (Torkzadeh and Doll, 1999) |
| Ap2 | This system will save you time | Survey | 1-5 | (Torkzadeh and Doll, 1999) |
| Ai1 | Would it increase your task innovation? | Interview | 2-4 | (Davis, 2004) |
| Ai2 | This system will help you come up with new ideas | Survey | 1-5 | (Torkzadeh and Doll, 1999) |
| Au1 | Would it be useful in your job? | Interview | 2-4 | (Davis, 1989) |
| Au2 | Using this system would make it easier to do your job | Survey | 1-5 | (Davis, 1989) |
| Nt1 | Would this technology fit your job tasks? | Interview | 2-4 | (Davis, 1989) |
| Nt2 | The data in this system will be current enough to meet your business needs | Survey | 1-5 | (Davis, 1989) |
| Ne1 | Would this technology be easy to use? | Interview | 2-4 | (Davis, 1989) |
| Ne2 | Learning to operate this system would be easy | Survey | 1-5 | (Davis, 1989) |
| Nq2 | Would this system have sufficient data quality? | Interview | 2-4 | (Erlingsson and Grødem, 2004a) |
| Nq1 | The data in this system will be reliable | Survey | 1-5 | (Doll and Torkzadeh, 1988) |
In order to use these measures to investigate the
gap between the IS/IT experts and the end-users, suggested in the
research model, six measures, at three different points in time, where
done. The first two were measures of the IT/IS experts after the
creation of a feature list for the system but before they creating the
scenario. The two next measures were also of the IS/IT experts, but
after the individual scenario creation, but before the creation of the
joint scenario. The final two measures were done of the end-users after
they had read the scenario and taken part in the interview. The idea
behind this research is that the IS/IT experts will have a score closer
to the end-users after they have created the scenario, thus indicating
that the perceptional gap is closing. Therefore there is an obvious
limitation to the above number of measures namely that the end-users
are only measured once as will be discussed later in this book chapter.
The interviews were taped, after asking permission.
The research was also briefly introduced; not mentioning the
intentions, the research and interviewees explained how questions about
the scenario were not allowed. Questions to the interviewers about the
scenario would influence the interviewees understanding of the scenario
and as such, the answers. After each interview, a reflection session
was held, not to implement any changes but in order to solve problems
if found while conducting the interviews. Neither reflection session
uncovered any real problems in the interviews or the process.
The transcript and coding structure were discussed
after the interviews and were not strict. The coding method was decided
to be to use the comments feature in the word application, following
the already introduced coding of the questions outlined in the Research
model chapter. The coding of questions was only done on the interview
section of the interview guide, since that was the only part open for
discussion and interpretation. This coding and a transcript of the
interviews were found to be suitable for the intentions of the study,
the limited number of interviews and because the researches already
used survey measures to some extent. Obviously more interviews would
have demanded stricter coding and transcript policies, due to the
complexity of analysing a large number of interviews, the quality of
the work and the often overwhelming task of analysing data which is
lacking early coding and analysis of data (Miles and Huberman, 1994).
The final scoring of the interview, for all the interviews, was done by
studying the transcripts shortly after each interview.
All the answers, the first and second rating of the
IT/IS experts and the users, were combined in a table in order to
compare them easily (Erlingsson and Grødem, 2004a), which can be
found as an appendix to the book chapter. The answer in the appendix
are the average rating of the IT/IS expert group and the end-user
group, in accordance with the intention of the research.
The population of choice is a large Nordic IT consultancy company in which the authors, as previously mentioned, have conducted previous research in. The reason for choosing this organisation is based on the fact that the authors is building on- and investigating and developing an idea which was conceived during this research. The sample size is two individuals (consultants) within the above organisation. The two individuals were chosen based on the authors knowledge of the interviewees, the belief on how the position they hold in the organisation were well suited for the research carried out and to the fact that the nature of their work is believed to correspond with the idea developed for the organisation in previous research.
In order to analyse the research model and to answer the research question some criteria had to be set. Two methods will be used in this study, one is analysing the codification (the scoring) of the interviews and the surveys and the other is by analysing the interview transcripts. The first method of comparing the scoring is simple and gives a first indication of each interviewee’s perception. The second method of analysing the transcripts, while more demanding and difficult to compare initially, contains more data and has the possibility of giving in-depth understanding of the underlying reasons for why the interviewees is answering one-way or the other.
The analysis of the model based on the scoring was
done in two steps. Firstly, looking at the end-user score for scenario
use and secondly by comparing the end-user score for each construct
with the IS/IT experts scores before and after the scenario creation.
In order to say that the authors were successful in creating a useable
scenario, the ‘scenario use’ construct was decided to have a score
higher than 3 (3 = Neither agree or disagree) on each of the variables.
This is derived from the fact that if the end-users disagreed to the
scenario constructs no useable scenario was created. This would mean
that the research question could not be answered under any
circumstances. The experts would then simply not have created the
intended scenario, but something else, which users and scenario theory
would not have recognised as a scenario. In order to accept the
propositions the dependent construct scores of the IS/IT experts, after
having created the scenario, should be closer to the end-users than the
scores before the scenario creation. It can therefore be derived that
if only one of the propositions were supported this would be sufficient
support for the research model.
The analysis done based on the transcripts was intended to uncover some reasons for the answers given. Trying to uncover if there is a similarity between the end-users and the experts, which is not indicated in the scoring, will possibly support or reject the research model. Another aspect was to try to finds flaws in the research model, which indicates that improvements that can be done.
Obviously this methodology or research design has limitations. The most obvious one is that the end-user is not measured twice. Another limitation is that other influences, rather than the scenario use, cannot be ruled out. Furthermore, the research is not investigating if any other method, than the use of scenarios would have given the same results.
The analysis part is divided into two parts. The
first part covers findings and the second part covers the discussion
and refinement of the research model.
The findings, based on scores, would reject the research model.
However, analysis of the interview transcripts indicates that there is
more nuance to this conclusion. The experts might have been overly
positive- and the end-users conservative in their rating. Thus the
end-users and IT/IS experts are further apart than need be. Moreover,
the analysis indicates that there is a positive movement in the
end-user perceptions about the system and indications of end-users
moving closer to the IT/IS experts in the way they answer.
Initially only the scoring was used to analyse the research question and the research model. Based on these scores Table 2 was created to ease the analysis. Firstly, the independent variable was analysed, by asking the question if the scenario was successfully used. Based on the criteria given, the scenario was successfully used and all variables scored higher than 3. This indicates that the scenario used was successful in creating the desired effects on the dependent constructs and therefore the measurements of those constructs can be used. In order to accept the research model either one of the dependent constructs would have an average IS/IT expert score, after the scenario creation (second interview), closer to the end-user score than before the scenario creation (first interview). Since neither dependent construct met these criteria, the research model has to be rejected, within the limitations of this study.
Table 2 IS/IT Expert and End-user score
| S. Scenario use | A. Individual actor effect | |
| Sd. End-user average score 3,5*
Sp. End-user average score 3,5* Sc. End-user average score 3,75* |
Average end-user score: | 25 |
| Average IS/IT Expert first interview score: | 26 | |
| Average IS/IT Expert second interview score: | 27** | |
| N. Information needs and requirements effect | ||
| Average end-user score: | 22,5 | |
| Average IS/IT Expert first interview score: | 21,5 | |
| Average IS/IT Expert second interview score: | 25,5** | |
*Higher than 3 is acceptable
** Should be closer to Average end-user score than Average IS/IT
Expert first interview score
Having said this, the analysis of the research
model based on the transcripts gives a different result than the
analysis of the scoring. The analysis of the interview transcripts
indicates that this is partially due to the fact that the end-users are
found to be more conservative in their scoring than the IT/IS experts
in their second interview. An example is how both one of the experts
and an end-user name information overload as their primary concern. The
IT/IS expert states: “the information overload challenge must be
taken care of” – IS/IT expert (Grødem, 2004) in the second
interview and the end-user states: “Information overload, constant
demands from the system”
- End user (End-User-J-Interview, 2004), in questions regarding
individual actor effects. The “information overload” statement has no
effects on the IS/IT expert answer. The IS/IT expert still gives a
maximum score for that construct. The end-user on the other hand rate
that construct lower than the IS/IT expert, based on the information
overload.
Another consideration is the difference in the way
the end-users and the IS/IT experts answer the questions in the second
interview. The IS/IT experts are very positive in all their comments of
the ideas after the scenario creation, the only negative comment is the
information overload comment previously mentioned. In the interview
before the scenario creation, the answers are a lot more sceptical
focusing both on positive and negative effects. This is more in line
with how end-users answered the questions in the interview. Since the
effect of the negative comments can clearly be seen how the IS/IT
experts scores in the first interview and the IT/IS experts’ second
interview has few negative comments of the system. This indicates that
there might be in the IT/IS experts’ answers in the second interview,
after the scenario creation.
Since end-users are believed to be more
conservative than IS/IT experts in their second interview, and it seems
like the the IS/IT experts are overly positive, it can be assumed that
the gap between the IS/IT experts and the end-users is over estimated.
This indicates that the research model can not be rejected as quickly
as done by only focusing on the scores.
The above findings indicate a more complex and nuanced picture of the
use of a scenario to reduce the perceptual gap. In the further analysis
of the interviews the authors therefore focused on trying to find
indications of how the scenario has reduced the gap between the
end-user and the IS/IT experts by trying to find statements that
indicated a move of either group closer to each other. There is one
indication of movement to be found. That is found by analysing the
interview in light of the ‘scenario use’ scoring. End-users score
higher than 3 on all the scenario use variables it indicates a move.
I.e. they agree that the scenario has changed their vision of the
future. Furthermore, in the interview they admit that this is a
beneficial future; “This is something I really would like to have” (End-user-H-Interview,
2004), “I would be more efficient, finding checklists, using more
criteria’s…get more short information and having some level of rating
would help” (End-User-J-Interview, 2004). The above statements
show how the end-users had not thought of this before (thus, a new
future) and how the system would be beneficial indicates that the
scenario has resulted in higher scoring on behalf of the end-users.
This can signify that if the end users had been asked before the
scenario creation they answers would have answered differently. In this
case this would be how users-would have scored lower than the IT/IS
experts at that time and thus might have a bigger perceptual gap at
that point in time than after being presented with the scenario.
However, this does not make it possible to accept the research model,
but indicates that if the end-users had been asked based questions
based on the feature list alone they would likely have scored lower and
thus the perceptual cap could have been bigger at that point in time
than after being presented with the scenario. Still this can only be
forwarded as a speculation and needs further research including
end-user interviews before the scenario is presented.
Based on the above the findings are inconclusive since the first measurement of the end users is missing. However, the analysis supports the research method and this is believed to be viable method of measuring the proposed research question and model. Furthermore, after the interviews were conducted one interviewee stated “now [after the interview part] I would score the scenario much higher” This indicates how he has moved closer to the IT/IS Experts.
Even though the scoring concludes that the model
has to be rejected this is believed to be mostly due to methodological
reasons. Furthermore, even if two interviews for each measurement are
statistically invalid, a generalizeable result was never the intentions
of this study and conducting two interviews, means that an answer to
the research question and propositions would be valid only for these
two end-users. Still, as mentioned, the analysis indicates that there
are some problems inherited in the measurements and that it is probable
that the research model and thus the propositions would be supported if
the methodology used was different. This section is dedicated to
discussing this mismatch, trying to uncover why this happened and how
this can be amended. In the end a refined research model will be
introduced.
The analysis of the scoring and the transcripts
uncovered a number of issues that have been addressed in the analysis
section of this chapter. Two of those issues are closely related. The
first being the fact that the negative comments made by the end-users
have stronger effects on the scoring than the negative comments made by
the IS/IT experts in the second interview. The other issue is the fact
that the IS/IT exerts are overly positive when discussing the idea in
the second interview. Both of these indicate the possible bias in the
scoring, making the gap appear bigger than it is. The reasons for this
bias are most likely to be found in the scoring of the IS/IT experts
and the fact that they are the creators of the idea and the researchers
in this study. The idea is their creation and thus it is natural for
them to be overly supportive of the idea. One of the restrictions given
by the methodology of this study was that there should be as little
external effect on the scenario creation process as possible, resulting
in the authors, being the idea creators, had no external evaluation and
critique on the idea. As the authors can be said to be a homogenous
group (Morgan, 1998), both being IS/IT experts and have similar
educational and experience background, the absence of external critique
makes it very likely that groupthink (Greenberg and Baron, 2003) and
even Hubris can occur. (Karp and Jackson, 2004) develops. Hubris, being
to the individual what groupthink is to groups, can lead the group and
the individual, into failing to think critically, rejecting potentially
correcting influences of outsiders (Greenberg and Baron, 2003, Karp and
Jackson, 2004). In this study this can be one explanation of why the
IS/IT experts answer the questions as they do in the second interview.
Another explanation can be found by comparing the
final version of the scenario with the individual scenarios created.
When combining the two scenarios into one the decision was made not to
have the final scenario technical and focus rather on functionality,
thus taking out a lot of technical descriptions from the first
scenarios created. It can clearly be seen from the individual scenarios
created by the IS/IT experts that they have found solutions for many of
the problems the end-users focused on based on the final version of the
scenario.
There are number of issues related to the methodology used in this
study. The main issue is the lack of an earlier or pre-scenario
measurement of the end users. Within the scope of this study it was not
possible to measure the end-users twice and thus the only measurement
made was after they had read the scenario. This resolved in the
rejection of the model even though there are indications from the
transcripts analysis that they have moved from lower score to the score
they got in the second interview. The simplest way of solving this
would have been to measure the end users twice and thus have a clear
measurement of the perceptual cap and how it changes over time. As
stated earlier this was not possible due to limitations of this study,
but the authors have found some possible ways of dealing with this
issue without expanding the scope. The first possible solution would
have been to give the end-user the feature list first, ask the
questions and then give them the scenario and ask them again. The
problem with using this method would be how close in time the two
measurements are and thus they might influence each other. Another
solution would have been to expand and use the scenario questions in a
different way. In this research the questions for the ‘scenario use’
construct are only used to see if the scenario was successfully created
and applied. What is clear from the analysis is that they could have
been used to establish movement of perception and by extending the
questions to ask the end user how he feels that the scenario had
changed the perception compared to if he had only gotten a feature
list. The problem here is the difficulty of asking the end user how he
thinks his perception would have been different, which is a very
difficult thing to do. Of the two methods, the authors believe the
first will be more likely to provide valuable results.
Another issue in relations to the method used is
related to the scenario. The scenario was created to represent as many
of the company employees as possible; it should have elements
recognized by everyone. This way of writing scenarios is supported by
literature (Lindgren and Bandhold, 2003, Ringland, 1997), but in this
study the aim is to see if by writing the scenario the IS/IT experts
will gain more understanding of the end-users. Since the scenario is
very general its creation does not require the IS/IT experts to get a
deep understanding of the end-user individual needs and information
effects (the two dependent constructs), they only have to have a
general understanding. This leads to a questioning of the dependent
construct: Will the scenario creation have an effect on the perceptual
gap on the individual level, or should the focus of the study rather be
on a higher level, such as the ‘organisational structure and network
effects’ as outlined in Larsen (1999) .
Even though the findings where inconclusive two
variables in the research model were found to benefit from improvement
or even cancellation. The first is the ‘ease of use’ variable. It was
clear from the answers given by the end-users that it is very difficult
for end-users to perceive ease of use at this stage in the development
process. When asked they answered: “I am quite sure that it is
meant to be, I am a little afraid that if you go for it, it would not
be easy to use for long [time]” (End-User-J-Interview, 2004)) and “It
will be extremely difficult to make complicated system like this easy
to use”
(End-user-H-Interview, 2004). This indicates that even though they
believe it possible could be easy to use it will be difficult to make
easy-to-use system and that it will not happen easily. This also
indicates that the end-users would like to see more of the solution to
make up their mind and more than what is possible at this stage in the
innovation process. Therefore the research model would benefit from
excluding this variable. Another questionable variable is usefulness.
It also shows through in the interviews that by asking questions about
the other two variables, productivity and innovation, the end-users are
essentially answering if they find it useful or not. That leaves the
usefulness variable redundant. Therefore the variable should be
excluded from the research model.
Based on the refinement mentioned a new research model can be forwarded (Figure 2). Further changes of the research model could be possible if the research was redone and the findings were more conclusive. However, in the event of redoing the research the main recommendation is related to the methodology and not the research model. When redoing the research the researchers should either ask the end-user questions at one point in time based on a feature list and based on the scenario or they should ask the end-users at two points in time, once about the feature list and once about the scenario. The latter being the preferred method.
Figure 1 Refined Resarch Model
The above described research is carried out by
using different methods and theories. One of the methodological choices
made is how the researchers are involved in the research themselves.
This has implications for this study in which, as explained above, the
researchers becomes biased and influences the study. However, this is
in line with what could be expected, based on theory, and does
influence the results. This is a practical learning experience and
implication to the authors and researchers in how one can try to limit
this in future studies. If this was expected, having had previous
experience in action research, this might have been omitted and thus
influenced this study less. Another major implication of the study is
the realisation of how the first measure of the end-users was lacking,
or very much sought after. Having realised this at an earlier stage,
should have caused an alteration to the research design, trying to
capture this measure in one of the earlier mentioned ways. However, as
stated initially the authors are doing action research, but with no
alterations and iterations. Naturally, the next step would be to
iterate and change the methods, process, and measures and use this
knowledge gained to do the research again, with the aim of achieving
something which is more useful than what was achieved by this first
try.
Even if the suggested research question was not answered, and thus difficult to draw any profound conclusions from the researchers believe that scenarios can be used. The practical experience of writing the scenario was very useful. It is a totally new way of writing and thinking, as it forces the scenario author to think and write about systems which is plausible, challenging and makes the end-user and the experts able to make up their minds on implications, effects, benefit and uncertainties of the suggested new system. In addition to this effect and believed positive outcomes of using a scenario the authors came up with another area of applying scenarios with luck. This was the idea of using a scenario in all bids, or new customer offers. Specifically in the investigated organisation, but also companies in general. This idea was not mentioned to the organisation at any stage, but after the interview, one of the interviewees, which are the bid manager for a large current bidding process, had the same idea. The bid team will, for the first time in the company history, put a lot of resources into writing three different scenarios to the customer explaining and showing understanding of how the company (experts) has understood the customer (users) needs of a new proposed system. This, and the research done, has led the authors to become very certain that scenarios are useful in many practical areas, which demands an explicit description of the future, and therein reducing the perceptual gap between end-users and IT/IS experts.
As stated before there are limitations to findings in the research. The authors are certain that by conducting the same research again, corrected for found limitations and methodological challenges, they would have gotten a different result. A suggestion for future research would therefore be to conduct the research again by adjusting for the suggested improvements, in line with the action research theory, and thus get support for the research model. Thereby, by having established the needed initial support the next logical step would be to increase the sample and number of interviews in order to get generalizeable results
To our fellow students and professor in the BI, Fall 2004 course, GRA6645 MIS topics, for valuable comments and suggestions.
Both practitioner and academic literature continue to devote considerable attention to the issue of how information systems development can be improved through the application of new tools, techniques, principles and methodologies (ISDM) (Andersen et. al 1990). Although some authors describe the existence of a methodology jungle within the IS development field (Avison & Fitzgerald, 1995), Object Oriented (OO) development methodology has in the last years been recognized as one of the most prominent innovations in the IS development field.
This chapter tries to map the expected and realized benefits of the OO methods on the IS project level. Structured interview techniques are employed to identify the gap between the objectives and realized benefits of employing this particular development technique within IS projects. The interviews are conducted with employees from systems analysis and software development departments in one of the largest banks in Turkey. There has been a migration from structured development methods towards OO languages and development methods in developing web based applications in this bank for the last two years.
The next parts of this chapter will present a research question and a research model. Through the rest of the text, a research model is introduced to depict the IS effect model in context of OO development methods and IS project performance. Benefits and barriers for making use of the Object oriented methods and approaches are discussed.. This provides a theoretical background for the interview. Following this a structured interview guide for the project leaders involved in IS development efforts in the mentioned bank is presented. Finally, the results from the interviews are analyzed and the results are evaluated in a theoretical discussion.
The outcomes of using OO development methods are described as shorter development cycles, easier prototyping and modular architecture (Sircar et.al 2001). While the software development community has extensively adopted and benefited from Object Oriented Technologies, using programming languages like Java, C# or C++, OO development methodologies are used in a limited number of the projects as the main IS development method (Glass 1999).
Shorter development cycles, easier prototyping and modular architecture should logically be preferred in software development projects, and it seems like a paradox that OO technology is widely used, but OO development methodology is not. The aim of this paper is therefore to find if using OO development methods will have positive effects on systems development project performance. Thus, the research question is:
“Will the use of Object Oriented Development Methods increase the IS project performance?”
A positive relationship between the use of OO development methods and project performance will mean that companies that still use structured software development methodologies can expect positive benefits from changing to an OO development methodology.
The following part of the chapter will discuss project performance and how to measure this. The difference between structured development and OO development will also be discussed, before presenting a research model based on the research question and the theoretical discussion.
Delivering the value of information systems is in general hidden within the success of IS development projects. Many of the tasks and functions within the IS departments are organized as projects. Cleland and King (1983) define a project as a complex effort to achieve a specific objective within a schedule and budget target, which typically cut across organization lines, is unique and usually not repetitive..
A classical comment on IS project tasks forwarded by Gray and Larson (2002) states that whether the goal is to design, install or reengineer, the project initiatives are often driven by aggressive deadlines and periods of frequent change. Under this tight schedule and frequent change expectations, measuring the success of the project and evaluating the outcome become an important issue.
Karlsen and Gottschalk (2003) proposed a five-dimensional approach to evaluate an IS project performance. The basic points in measuring the success are presented as follows:
It is apparent that the presented project success criteria use the whole organization as an evaluation background. However, as the focus of this paper is directed towards analyzing the success of IS development project at an IS project level and IS team, using project performance and project outcome as the evaluation criteria will be most relevant in the context of this research.
Project management literature often intertwines the project management success and the product success (Baccarini 1999). Conceptually, project management success has disregarded product success, meaning that even though the product does not meet user requirements, the project is a success if it has been well managed (Shenhar, Levy & Dvir 1997). According to Baccarini (1999), project management success and product success is evaluated by the following criteria:
Project management success:
Product success:
These two different views can be merged to see the
whole picture related with the success of a project in an
organizational setting. Figure-1 illustrates the different dimensions
of project success.
Figure 1: Project success related to ISDM
The introduction of a new IS development method may affect several of the criteria shown above, but not all directly. Since the aim of this research is to identify the effects of a change in “IS level” (Larsen 1999), the project performance is measured using the criteria relevant to this level. Since the identified project success criteria falls under what is covered in IS user element, IS application and IS platform (Larsen 1999), the organizational aspect can be left out in this research. By removing the organizational criteria and focusing on the criteria that would be directly affected by an IS development method, it leaves us with “Satisfying the users needs” and “Meeting time, cost and quality”.
“The users needs” can be defined as requirements in an IS development setting. Some of the requirements are translated into user-features, and the rest includes system interoperability and preservation of architectural integrity. The scope of the project is identified through a requirement-analysis in the beginning of the project. In this manner, the success in scope of the project can be measured in terms of meeting the user requirements and its compatibility with the rest of the system.
The “time, cost and quality”-criteria in an IS development project can be split up into two dimensions, resources and quality. Resources are in IS development projects the use of time and money. Time can be measured in terms of meeting the schedule (McCoy 1986; Morris & Hough 1987; Pinto & Slevin 1988; Turner, 1993). Success when it comes to cost can be measured in terms of meeting the budget (McCoy 1986; Morris & Hough 1987; Pinto & Slevin 1988; Turner 1993). The major component of cost in IS development projects is personnel cost, which is a function of person-hours spent (Rakos 1990; Kemerer 1996). Other costs related to the project are hard to measure, as these are often fixed costs and not directly related to the project itself. The resource dimension in software development projects are therefore measured by number of hours spent on the project.
Quality can be measured in terms of conformance to functional and technical specifications (Morris & Hough 1987; Baker, Murphy & Fischer 1988; Turner 1993). Quality in a software development project is a measure of in which degree one can maintain the code and how well the information system scales (Baskerville & Pries-Heje, 2004). Maintainability is important if the information system is supposed to run for a long time and other programmers must work with the code. Scalability is important when the system usage increases and other functionality must be added to the information system. Security is also included in this dimension.
The criteria for IS development project performance can in this setting be measured in terms of scope, resources and quality. These are shown in the figure below.
Figure 2: Dimensions of project performance.
The criteria for measuring project performance are
closely connected with each other. This relationship between the
criteria can be illustrated as a mathematical formula:
Figure 3: Relation between project performance dimensions (Q is quality, S is Scope, R is resources and P is a measure of project performance and should remain a constant).
This formula is neither empirically tested nor grounded in literature. The purpose is to show the relationship between the dimensions of project performance. The reasoning behind the formula is as follows. If everything in a project goes according to plan, there is no need to change anything or prioritize different. If the project is off track, the criteria must be adjusted. If a software development project has problems fulfilling all the requirements, one can either spend more resources (money or time), lower the quality or reduce the scope of the project. In the same way, if a software development project is off schedule, one can either spend more resources, lower quality or prioritise some of the requirements thus reducing the scope.
IS projects and development is a complex, costly, and high-risk endeavor. In order to manage complexity and mitigate risk, organizations invest heavily in tools, technologies, and methodologies associated with IS development (Sircar et. al 2001). These tools and methodologies affect the IS project performance by changing the inertia in project management techniques and the IS team.
Object Oriented development languages emerge as an innovation in the IS field where they promise shorter development cycles better quality/maintainability and less use of resources. By this nature, this particular way seems to be a means of increasing the project performance. Moreover, Jacobson (1994) and Kernel (1993) propose that use of Object Oriented Development Methods, particularly Unified Modeling Language (UML) will further increase the software quality and maintainability in the environments where object oriented development languages are used.
Jacobson’s (1994) and Kernel’s (1993) arguments make sense when the class based structure of the object oriented programming languages are considered. It can be argued that the superiority of OO development languages lies in the fact that they make it possible to re-use classes both within a single project and between different projects.
The class based structure of the OO programming languages enables the system developers to represent the real world better.. This structure helps to modularize the structure of the program. Different modules can be used to build prototypes easily thereby reducing the cycle times in each development effort. In addition, OO development methods, emerged after object oriented languages, helps to describe the IS project requirements in a more compatible and comprehensive fashion for developers using OO languages. They are used to document the requirements mainly by emphasizing on use-cases. Use-cases describe the relationships between the system and the users, as a sequence of related transactions performed by an actor and the system in dialogue (Jacobson et. Al 1994). After this step, the main objects in the system are identified both with their data attributes and process involvements. Following this, similar objects are classified to form classes/super classes according to their attributes, and a class diagram is constituted. The class diagrams establish the basic documentation with sequence and collaborative diagrams. The development of the system is then accomplished through combination and reuse of classes mapped in the documentation.
The unique way in analyzing the system and documenting the requirements with object oriented development methods enables both software developers and system analysts to accomplish the tasks in an IS project more tightly, faster and in an incremental fashion. (Lindstrom and Jeffries 2004) Furthermore, the usage of the class-based structure in analysis part of the system completely coincides with the OO language structure. Hence, it can be argued that OO development languages and the OO development methods are complementing each other and to exploit the full benefits of OO methodologies they should be used together.
Lindstrom and Jeffries’ (2004) argue that the usage of the OO languages and the development methods is one of the primary enablers for emergence and success of the Agile Development Methods (XP, SCRUM, Crystal, Lean Development). In this new perception, the pace and the quality of development is more important than everything as the customer requirements gets more complicated and the competition gets tougher. Taylor (2003) further suggest that Agile development methods are mandatory to meet the needs of today’s “agile” and knowledge based organization where achieving absolute project success in terms of reduced cost and time, increased quality and increased met requirements are eminent. Those arguments and the link between employment of object oriented development methods and materialization of agile methodologies further supports the idea of direct effect of employment of OO development methods on IS project performance.
Although the anticipated benefits of OO development languages and methods are numerous, the OO development methods have not been adopted widely as the primary IS development method. While the 39% of the organizations adopted OO technology in some form, only 5% of them employ OO development methods (Glass, 1999). On the contrary, structured development methods are still widely employed in many organizations where object oriented programming language is used. The barriers for using the object oriented development methods can be mapped on three dimensions: The lack of an expected structure in OO system development methodologies and structure and control promise of structured development methods on IS project level and different interests and the power structure of the stakeholders involving in system development task.
Firstly, Ling and Teo (1994) state that, even though the object-oriented approach is very popular, it has a number of inadequacies, e.g. lack of a formal foundation, lack of a standard query language and widespread education about it. Muller (1999) also adds that developers still use the relational databases to implement the objects to make them persistent with the previous work, making the OO development and documentation principles unused. Basically, these two points illustrate the basic barriers in adoption of OO system development methods. Furthermore, Sircar et al (2001) stresses on the fact that a migration from structural IS development to OO IS development requires a change in the general understanding of system architecture which should be handled carefully to achieve the full benefits of an OO IS development method.
Secondly, Hanakawa et al (1999) identifies one of the major problems in object oriented software projects as the lack of management’s ability to comprehend and control the development process of the project. Their reasoning behind this is the fact that traditional phases of software development are not appropriate for OO development. In contrary, the most known structured IS development method, the waterfall methodology, promise a traceable and fully structured way of conducting the IS development projects. This method starts with requirements definition and follows a sequence of analysis, design and implementation. Process models and data models are developed in analysis and design phases respectively. These phases are followed by implementation and maintenance. The basic characteristics of a structured development method can be classified as the process models and data models. In the structured development method, process and data models are independently modeled. Process models are represented by data flow diagrams capture the flow of control among processes and demonstrate how data get modified through the system. Data models are described using entity relationship diagrams showing data elements and their associations. Later, the conceptual data model developed during analysis is normalized and transformed into a database scheme. Simultaneously, the process model is transformed into structure charts showing the hierarchical relationship among the programs to be implemented. Detailed programming logic is also developed in this phase. The database is implemented using the database schema and populated with data during implementation. Application programs are then developed based on the structure charts and program logic. Finally, the IS is tested to verify that it meets the users requirements (Sircar et. Al 2001).
This highly structured approach yields benefits in mapping out the needs in large-scale projects where development phases, analysis, design and implementation are executed in a sequential fashion. Avison and Taylor (1997) state that waterfall model is suitable for well-defined problems with clear objectives and known user requirements. Furthermore, this traditional methodology is well suited for rigid organizations where the boundaries and relationships are also well defined. Although this method is criticized for being too dependant upon organizational control and too mechanistic to be used in detail (Nandhakumar and Avison 1999), it is still the preferred IS development methodology in practice.
A third barrier for the OO development methods might emerge from different interests of different stakeholders in an IS project. Alvarez and Salazar (2003) state that IS development processes are large and complex socio-technical phenomena and requires many inter-departmental relationships. A strengthening argument is also made by Sillince and Mouakket (1997), concluding that different shapes of power in an organization and the willingness to be in control of the IS project may lead to different set of interactions. They further state that the power is based on information and the owner of the information within the projects would be the winners and the controllers of the processes in a specific project Thus, a specific link may be established between the structure and the control promise of structured IS development methods and the willingness of the IS project stakeholders’ acquisition of the power depending on the information they have about the project.
The following three arguments may help to sum the theoretical discussion. First, the delivering value of IS is hidden within the success of IS projects, and the success of the project is highly dependent upon the specific development methodologies. Second, OO languages and development are expected to increase the IS project performance considerably in usage of resources, achieving quality and meeting the requirements since they provide shorter cycle times, better prototyping and modular architecture. Third, there are already some barriers, which may prevent the organizations from exploiting the expected benefits of the OO development methods. In light of this, the formulization of the research question is presented in the next section.
The main emphasis in the theory presentation and the research question has been to identify the relationships between usage of OO development methods and the project performance. Shortly, object oriented IS development methods are expected to foster great benefits in IS projects, leading to shorter development and analysis time, better maintainability and easier prototyping. In this part, a research model based on the previous discussion is presented.
The main variables in the research question can be
identified as “Usage of OO IS development methods” and “IS Project
Performance” per se. The independent variable “Usage of OO IS
development methods” contains three main constructs, which are
forwarded by different authors and discussed in theory presentation.
Hence, the main promises, the short cycle times, prototyping and
modular architecture attributes of OO development methods, emerge as
the constructs of the independent variable. The IS project performance,
the dependent variable holds three constructs; resources, quality and
scope. Thus, this brings the following research model:
Figure 4: The research model
The research question is tested on qualitative data. The operationalization area, data collection methods and the structured interview guide are presented in this part. The expectations from the interview results can be split in two. First, it helps to see to what degree the potential benefits of an object orientated IS development methods are exploited in a corporate setting. Second, it helps to identify barriers in a transition from structural development methods to an OO environment. The results are analyzed and presented in light to these expectations.
To test the research question and differentiate between the expected and the realized benefits of employing object oriented IS development methods, a relevant setting where structural IS development methods have been replaced with object oriented IS development methods should be selected. The researchers have chosen a system development team in one of the largest banks in Turkey. The IS development team changed the programming language from a structured development to Java three years ago. Although, they continued to use waterfall methodologies in managing the projects for one more year, a gradual move has been observed towards Object Oriented Methodologies. Specifically, the system analysis team started to map the data models and user requirements in Unified Modelling Language (UML) and convey these to the development team. The interviewees are selected among the project managers both from system analysis and development teams.
As the practical environment constitutes a suitable ground for the analysis, a semi-structured interview guide is used to identify how a transition to Object oriented IS development methods worked in practice. The main goal of the interview guide is to identify the realised benefits of OO development methods. Secondly, the barriers discussed in theory section are used to explore the change process in migrating to OO development method usage further. It also includes questions about the organization and the work context to describe the background. Section 5 in the interview guide is the part where the answer to the research question is sought. The rest serves to identify the organizational background and barriers. The Interview guide can be found in appendix 1.
The interviews were conducted through phone. Although the interviews were carried out in Turkish, there has not been any difficulty in communicating the terms related with development methods and the IS project performance as all the interviewees are using these terms in English in their daily routines. The interviewees are project managers from the System analysis and the software development departments respectively. After the interviews were conducted, their transcripts were prepared in English. The reciprocal translation method has not been used in this study, but as the interviewer has a Turkish background and has good knowledge in the IS field, this is not expected to pose any serious problem.
The compilation of the data from the interviews was not complicated due to the fact that only two persons were interviewed.. In the analysis, all replies to the questions were analysed and a selective process was not used to differentiate between relevant and irrelevant data as the all the responses were thoroughly compiled. The interview guide was designed with some closed-ended questions directly related to the research model. This, the limited scope of the research project and the fact that there were only two interviewees, made it possible not to use protocol analysis during the analysis phase. The interview data were instead thoroughly analysed by using the answers from the open-ended questions related to the organization as a background for the answers to the research model related questions.
The data analysis intends to derive the following main points out of the responses. First, the data specific to the organizational background is filtered. This helps the authors to evaluate the answers to the research question on a wider basis. Secondly, the responses to questions related with the research model are evaluated. Thirdly, the data about the possible barriers are identified to support the final discussion and the conclusions.
The interviewees work in the IT department of one of the largest banks in Turkey. The bank as a highly information sensitive entity, has put high emphasis on IT investments throughout the years and achieved high efficiency in operations and attracted new customers through the quality of its alternative delivery channels.
The scope of the interview covers two main workgroups within the bank, the system analysis department and web based application development departments respectively. The tasks of the system analysis department (SA) include requirement analysis, documenting the system specifications and quality assurance. On the other hand, the web based application development team (WBAD) is responsible for developing and maintaining Internet banking and call centre applications. Furthermore the software development team has the responsibility of fitting the architecture of their applications with the rest of the infrastructure and the legacy systems. The SA and the WBAD can be seen as the two components of a value chain where the WBAD team use the output of SA as an input in understanding the requirements. Furthermore, usually the two departments use informal meetings and e-mail communication to clarify the requirements further.
The size of the IS projects involving the two departments vary, but they usually define tasks lasting 3-5 days as ‘ad-hoc work’ and the rest as projects. Although the projects may last for 6-12 months the average size of the project is about 3-6 weeks. Both SA and the WBAD teams complains about the continuous changes in the business requirements, which may even result in terminating the current project and start up a new project from the scratch. In this sense, the interviewees emphasised the ill-structured nature of the problems to be solved. The communication with business lines and the coordination of the IS projects are assumed by the SA department, though each department in itself operates autonomously. Being a mediator between the business lines and the software developers, SA holds a significant amount of power in the organization.
The technology used in the organization has undergone a serious change within last 5 years. The basic applications in the bank have been re-written using the component-based approach. The component-based approach inherits the basic benefits of OO technologies in many senses, but the programming language does not have to be an OO development language. The change process from the traditional programming approaches to component-based technology is still going on in the organization. The interviewees mentioned that 40% of the system still works on PLI and Cobol, which are difficult to change in short times. The WBAD team started using pure object oriented methods and Java in their applications three years ago. The software development teams and the research and development section of the IT department mainly carried out the selection processes. Although the SA department had not been involved in this decision process they were highly affected by this. The department had to re-specify the requirements for the parts to be re-written with new approaches.
OO-based languages and approaches have been used in the organization for a considerable time. Even though there has been an attempt towards using the OO development methods like UML in writing the specifications, the usage of this was soon dropped. The OO development efforts seemed to show different effects on two complementary departments. Hence, it would be useful to analyse the expected and realised benefits of OO development methods regarding the usage in two different settings.
The software development team initiated employment of OO methodologies. The data from the first interviewee reflected much on expected and realised benefits of OO methodologies regarding the research question. The WBAD decided to employ OO methodologies based on the following criteria, which created a base for the expectations in a software development setting.
Indeed, the scope of the expectations on WBAD part was quite wide. The expectations listed above, cover the constructs of the independent variable, shorter development cycles (10, 15), modularity (7, 8, 3, 5) and easier prototyping (16). Analysing the expectations on the dependent variables, the project performance, the WBAD had also strong expectations about a possible increase in the project performance. Since their previous method (Development in C using CGI protocol) became obsolete through the years, they thought the benefits of using an Object Oriented development language would boost the project performance.
Table-1 shows the expected and realised benefits of the OO development languages depending on the responses got from the Interviewee-1 (I-1).
| Question | Expectations of I-1 | Realisations of I-1 |
| (Did) OO development methods decrease the cycle times in the projects | Yes | Yes |
| (Did) OO development methods facilitate the prototyping in the projects | Yes | Yes |
| (Did) OO development methods help you to reach a modular architecture | Yes | Yes |
| (Did) Short cycle times decrease the usage of resources | Yes | Yes |
| (Did) Short cycle times increase the quality | Yes | Yes |
| (Did) Short cycle times increase the scope of the projects | Yes | Yes |
| (Did) Easier prototyping decrease usage of resources | Yes | Yes |
| (Did) Easier prototyping increase the quality | Yes | Yes |
| (Did) Easier prototyping increase the scope of the projects | Yes | Yes |
| (Did) Usage of a modular architecture decrease the usage of resources | Yes | Yes |
| (Did) Usage of a modular architecture increase the quality | Yes | Yes |
| (Did) Usage of a modular architecture increase the scope of the projects | Yes | Yes |
The analysis of the interview questions shows that the project performance of the WBAD team increased dramatically. In addition, I-1 stated that, “Previously, we were the bottleneck station in the process. Now, we can handle the requirements much faster than before. I guess, now the bottleneck part is the system analysis team in analysis and in testing” The resource usage decreased by 30-50% (depending upon the nature of the project), the scope of the projects enlarged hence, they can handle the new needs better and without ceasing the flow of the project. Furthermore, the easy prototyping enabled them to communicate with the end users to understand the needs better in an ill-structured project environment. Based on these data, it can be stated that our hypothesis is strengthened for usage of OO development languages.
On the other hand, the responses of the second interviewee (I-2) change the direction of the analysis. The priorities and the expectations from the usage of object oriented methods, particularly UML, is different than the WBAD team. The data from interviews reveals that the UML usage first suggested by the WBSD. The I-1, commented that, “It would be much easier to communicate what we were doing. Because, after we changed the development method, we were analysing everything one more time to fit the incoming analysis to the current structure, this is nothing but the duplication of the work, strictly, provided with only the process flow is not enough.” Following the suggestions of WBAD team a workgroup was established with members from WBAD and SA teams. After three months of pilot usage of UML methods on a medium sized project the change process terminated without seeing the solid benefits. Although the team experienced a steep learning curve about UML applications and UML users and programmers were quite sure about the long-term benefits of UML usage, I-2 states “The short term negative benefits of applying this method, overrode the long term sustainable benefits.”
The perceived short term negative benefits of UML by I-2 was listed as follows:
As can be inferred from the quotations from the
interview, the use of UML became questionable because there were
short-term negative benefits of it Furthermore, the long-term benefits
did not justify the perceived losses in the short term from the SA
manager’s viewpoint. The answers to the questions related with the
research model are presented in table –2.
| Question | Expectations of I-2 | Realisations of I-2 |
| (Did) OO development methods decrease the cycle times in the projects | Yes | No |
| (Did) OO development methods facilitate the prototyping in the projects | Yes | Yes |
| (Did) OO development methods help you to reach a modular architecture | Yes | Yes |
| (Did) Short cycle times decrease the usage of resources | Yes | No |
| (Did) Short cycle times increase the quality | Yes | No |
| (Did) Short cycle times increase the scope of the projects | Yes | No |
| (Did) Easier prototyping decrease usage of resources | Yes | ? |
| (Did) Easier prototyping increase the quality | Yes | Yes |
| (Did) Easier prototyping increase the scope of the projects | Yes | ? |
| (Did) Usage of a modular architecture decrease the usage of resources | Yes | No |
| (Did) Usage of a modular architecture increase the quality | Yes | Yes |
| (Did) Usage of a modular architecture increase the scope of the projects | Yes | ? |
Although the table lacks some answers to the critical questions, the following results can be inferred in combination with the quoted items from the second interview. First, in short term usage OO development methods do not deliver the expected benefits. Second, although the overall expectations are positive towards using OO development, the realised benefits may suffer from the organizational setting, workload and personal mindsets in decision making. Third, the data gathered from the second interview might be biased due to the initial time consuming experience with usage of OO development methods during the analysis process.
To sum, the analysis of the data from the research reflects the effects of the OO development methods on the IS project performance from the two sides of the same mirror. WBAD team felt the effects of this innovation in the projects substantially such that, although the SA did not use the corresponding OO based analysis tools, the overall project performance increased because of the OO development languages.. Although, this does not support the hypothesis completely, the segregation of the expectations and realised benefits at the organization level fosters several important questions, which are discussed in the next section to indicate the directions for the future research.
While a connection between the partial use of OO development methodology and higher project performance is significant at the software development side, the decision to drop the UML usage in system analysis side was just as interesting. The barriers presented in the theory section can partially help to explain the abandonment decision. Besides, this phenomenon can be explained with the following concepts derived from the organizational theory: Power structure and conflicts, personal beliefs, organizational culture, and organizational inertia.
The theory presented on the barriers of OO development methods is mostly grounded on the unstructured nature of OO development methods (Hanakawa et al 1999; Ling & Teo 1994; Muller 1999) and in contrast the control and structure promise of structured system development methods (Nandhakumar & Avison 1999). However none of the interviewees mentioned any facts related to those theories. On the other hand, implications of the fact that deep experience and knowledge of pros and cons of structured methods render it difficult to change, is worth to analyse using different perspectives.
Another overlap with the presented theory may surface from the Sircar’s (1999) arguments; a migration from structural ISD to OO ISD requires a change in the general understanding of system architecture which should be handled carefully to achieve the full benefits of an OO ISD method. The perceived long and painful change process was one of the reasons in dropping the UML usage. Education needs of the employees, a perceived slow down in the processes might be regarded as the symptoms of the same diagnosis.
The background questions from the interviews suggested that the SA department is the controller of the project and acts as a bridge between software development team and the business lines. Through this position, the department holds the power to control the projects and define the requirements (Sillince and Mouakket 1997). A change in the method of analysis would change the equilibrium position (Pfeffer 1981) between the departments as the software developers would be better off in grasping the necessities in UML and Object oriented methods, leaving SA in a redundant position at least for a period of time. Since the SA department may not want to lose its power on the business units and software development teams, it may get the organization to revert back to the old structured development method.
Links between the presented theories on the barriers for the usage of OO development methods are now established, but further theories might shed more light into this area. Schein (1996) claims that a majority of change programs fail due to the multiple cultures that exist within organizations and the lack of alignment between them when implementing change or adopting new work methods. It can be asserted that the different cultures and mindsets within the bank did not have the same expectations about the change, and this lead to difficulties in embracing the new methods for the organization as a whole.
Another important explanation to this phenomenon might be derived from the theories about organizational inertia. This theory claims that organizations are typically unable to match structural changes to their environments in a timely fashion (Ruef 1997). The results from the researches related to this theory suggest that organizational change increases the hazard of organizational failure in the short run but that this hazard wears off over time. The studies remain open to the possibility that the content of change may provide long term success. As the interviewees suggested, the initial cumbersome work and the perceived initial failure might have prevent the adoption of the usage of this method.
While the discussion on the organizational structure illuminates the drop of UML usage, the ‘project performance’ dimension in the research model did not take organizational aspects into account, as it was predicted that the methodology change would primarily affect the IS project level. It was assumed that benefits of using OO development methods on a IS project level would outweigh any negative effects on the organizational level. This was, in retrospect, a short-sighted constraint. Indeed organizational setting (possible resistance from employees, the mandatory change in the mindsets of the employees, initial time loss and communication concerns with business lines, power structure, subcultures) and the ongoing workload seem to build the high barriers for UML-usage in the system analysis department.
Although the discussion leads to a better understanding of the whole concept underlying the development methods and the IS project performance this could be carried upon a wider extent to test the presented arguments empirically. Therefore, the following hypothesis can be developed leading the research model presented in figure –5.
Figure 5: An extended research model.
Although new research model is large and contains many complicated constructs, it is useful to get the full picture and summarize the issues raised in the discussion part. The authors believe that deriving different research questions from the research model using the different constructs from organizational structure in IS setting will both open new areas for the aspiring researchers and simplify the model to render it doable.
During the research and the analysis, several limitations have been identified. The first point might be the number of interviewees and their positions. Even though the limited number of interviewees facilitated to carry out the interview and analysis tasks, this limited number may put a serious bias on the research results. Secondly, the interviews were carried out within the same company, in interrelated and complementary departments. Having one interviewee from the systems analysis department and the other from the software development department assisted to understand the whole picture about the expected and realised benefits of OO development methods. However, carrying out the same research across a number of different organizational settings may not only increase the reliability of the inferences but also change the direction of the results.
The research is conducted using a deductive approach. After the introduction of the research question, the related theories are presented. The discussion of the theories is limited according to the scope of the research question. The research model and data collection parts were built upon the theoretical discussion. Yet, the interview results induced the authors to think on a wider basis to explain the results and use new theories to refine and improve the discussion on drop of UML usage. Although the inclusion of new theories broadened the horizon of the research and helped to come up with a more comprehensive research model, the discussion of the theories is still limited due to space and the time constraints. Hence, the discussions and the theoretical reflections should be employed in different research projects to get the full benefits of these.
Throughout the text, expected and realised effects of OO development methods on the IS project performance is investigated.. A research model grounded upon the previously developed theory is presented. To support the arguments behind the research model, issues like measuring project performance and IS project concept, expected benefits of OO development methods and the barriers in front of the usage are discussed. The answers to the research question have been collected through semi-structured interviews. Through the analysis of the qualitative data, a link between the usage of object oriented systems development methodology and better project performance on the IS project level was found. Furthermore, the following implications for the future researchers are gained. First, The adoption of a new methodology in systems development might benefit the development process, but this does not necessarily mean that the organization is willing to embrace the new methodology. The short-term cost of adopting the new methodology might not outweigh the long-term benefits, but it is still a risk, and the organization might not be willing to take that risk. Second, organizational inertia, subcultures and power structure may also play an important role in resistance to the usage of OO development methods. Finally theory refinement and the research findings a broader research model is introduced to illuminate the paths of future research.
