Process Data Using Single Thread or Multithreaded

Process Data Using Single Thread or Multithreaded

Overview

For this project you will implement a program that processes files that represent purchase orders. The program can complete the processing by using a single thread or multiple threads.

Processing of Orders (Files)

Your program will process a set of files each representing a purchase order. Each file lists the items bought and the date of purchase. The possible items that can be purchased (along with the item’s price) can be found in a item’s data file (e.g., itemsData.txt). The program you need to write will generate a summary for each order (file). The summary includes the client id and a sorted list (by item’s name) of each item bought. The list will include the item’s name, the cost per item, the quantity of items bought, and the total cost associated with the item’s purchase. After the sorted list, an order’s total will be displayed.

In addition to a report for each order, the program will generate a summary of all orders. The summary will display a sorted list (by item’s name) providing information about the total number of items sold, and total revenue.

Threaded Processing

Your program will allow users to process all the orders using a single thread or one thread per order (file).

In order to see the advantages of threading, your program needs to print (to standard output) the time (in msec) it took to process orders. You can compute the time as follows:

longstartTime = System.currentTimeMillis();

/* TASK YOU WANT TO TIME */

longendTime = System.currentTimeMillis();

System.out.println(“Processing time (msec): ” + (endTime – startTime));

Driver

Your are free to define any number of classes/interfaces you understand you need, however, you need to provide a class called OrdersProcessor in the processor package. This class can have as many methods as you want, but it must have a main method that allow us to configure/run the processing of orders.

Your program will ask users how to configure a particular processing of orders by using standard input and output.

Enter item’s data file name:  itemsData.txt

Enter ‘y’ for multiple threads, any other character otherwise: y

Enter number of orders to process: 3

Enter order’s base filename: example

Enter result’s filename: resultsExample.txt

Reading order for client with id: 1003

Reading order for client with id: 1001

Reading order for client with id: 1002

Processing time (msec): 51

Results can be found in the file: resultsExample.txt

Requirements

  • You need to define a map/ArrayList that is shared and modified by all threads. How you use the map/ArrayList is up to you.
  • Each order has an item name, followed by the date (e.g. 6/13) the item was ordered. That date is not used in your program.
  • For the multiple threads option, one thread will be created for each order (file) to be processed.
  • You will start the timing process immediately after reading the name of the results file and will stop the timing process before printing the message indicating where results can be found. Make sure you close the file output stream for the results file before you stop the timing process.
  • Your program will print (to standard output, NOT to the results file) the message “Reading order for client with id: ” followed by the client id, before reading a particular order’s file.
  • Notice that the reports appear sorted by client id, but the processing of each order (when using multiple threads) is up to the Java scheduler.
  • The summary for all the orders only includes items that have been bought.
  • You may not use classes that are automatically synchronized (e.g., Vector).
  • Synchronizing individual methods is fine (in this case you many not need to use an explicit lock object), however, be careful as you don’t want to limit concurrency while trying to avoid data races. Instead of defining a method as synchronized, you should surround the critical section with a lock.
  • What we are expecting for the synchronization part of this project, is that any object (e.g., map, set, whatever you use) that is being shared is not associated with a data race.
  • Feel free to define as many classes and/or interfaces you understand are necessary.
  • Your program must be able to process any number of orders (not just 3).
  • Do not use static variables; do not use static objects. We want to avoid the JUnit problems discussed in class.
  • Your code must be efficient and you must avoid code duplication.
  • The report for each client must be sorted by client id.
  • You can assume that the specified output file (e.g., txt) will be overwritten (no appending of results) if the same name is used several times.
  • There could be items in the items’ data file that no one buys.
  • You can assume orders will only have items present in the items’ data file (no invalid data).
  • The file names will always start at 1 up to including the number of orders.
  • The performance of your program using multiple threads should be better than using a single thread.
  • Remember to follow exactly (except for spaces) the format associated with the file txt.
  • To print currency use getCurrencyInstance().format(AMOUNT_HERE). 

Solution

OrderProcessor.java

package processor;

importjava.io.BufferedReader;

importjava.io.FileNotFoundException;

importjava.io.FileReader;

importjava.io.IOException;

importjava.text.NumberFormat;

importjava.util.Map;

importjava.util.TreeMap;

public class OrderProcessor {

private String baseName;

/* Order detail of all clients, including summary */

privateStringBuffer[] finalMessage;

/* <item name, Data<quantity, cost per item>> */

privateTreeMap<String, Data>itemsMap = new TreeMap<String, Data>(); // Synchronization

// lock

/**

* Standard Constructor

*

* @parambaseName

* @paramitemName

* @paramnumOrders

*/

publicOrderProcessor(String baseName, String itemName, intnumOrders) {

this.baseName = baseName;

/* +1 for 1 indexing, +1 for total summary. */

this.finalMessage = new StringBuffer[numOrders + 2];

/*

* InitateitemsMap with complete items list, 0 for quantity, and corresponding

* price.

*/

try {

BufferedReaderitemFile = new BufferedReader(new FileReader(itemName));

String line;

while ((line = itemFile.readLine()) != null) {

this.itemsMap.put(line.replaceAll(“\\s+\\d+.*”, “”),

new Data(0, Double.parseDouble((line.replaceAll(“[^\\d.]”, “”)))));

}

itemFile.close();

} catch (IOException e) {

e.printStackTrace();

}

}

/**

* Note: called by main method from OrdersProcessor.

*

* @return finaMessage fully appended

*/

publicStringBuffer[] getMessage() {

returnthis.finalMessage;

}

/**

* Return a BufferReader that is ready to be read. Note: not closed.

*

* @paramfullFileName

* @return BufferedReader

*/

privateBufferedReadergetFile(String fullFileName) {

try {

return new BufferedReader(new FileReader(fullFileName));

} catch (FileNotFoundException e) {

e.printStackTrace();

}

return null;

}

/**

* Process the order associated with the order number given. This method creates

* individual message tailored for this particular client, and adds the message

* tofinalMessage array. Synchronized: appends itemsMap. NOTE: should be called

* beforeprocessSummary.

*

* @paramorderNum

*/

public void processorOrder(intorderNum) {

/* Get ID of client. */

BufferedReader file = getFile(baseName + orderNum + “.txt”);

String ID = “”;

try {

ID = file.readLine().replaceAll(“[^\\d.]”, “”);

} catch (IOException e) {

e.printStackTrace();

}

System.out.println(“Reading order for client with id: ” + ID);

/*

* Read and insert this client’s order information into local TreeMap

* (clientMap) AND insert the client’s information into global TreeMap

* (itemsMap).

*/

TreeMap<String, Data>clientMap = new TreeMap<String, Data>();

String line;

try {

while ((line = file.readLine()) != null) { // while still readable

String itemName = line.replaceAll(“\\s+\\d+.*”, “”);

/* Insert into local TreeMap (clientMap). */

Data count = clientMap.get(itemName);

if (count != null) { // increment the quantity if the

// item already exist in the tree

count.increment();

} else { // insert new one otherwise. NOTE:

// slow algorithm!

clientMap.put(itemName, new Data(1, itemsMap.get(itemName).getSingleCost()));

}

/* Insert into global TreeMap (itemsMap). NOTE: accessed by multiple thread */

synchronized (itemsMap) {

itemsMap.get(itemName).increment();

}

}

file.close();

} catch (IOException e) {

e.printStackTrace();

}

/* Append this client’s order information into a StringBuffer. */

StringBufferclientOrder = new StringBuffer();

clientOrder.append(“—– Order details for client with Id: ” + ID + ” —–\n”);

Double total = 0.0;

while (!clientMap.isEmpty()) {

Map.Entry<String, Data> entry = clientMap.pollFirstEntry();

int quantity = entry.getValue().getQuantity();

String item = entry.getKey();

Double perCost = entry.getValue().getSingleCost();

total += quantity * perCost;

clientOrder.append(“Item’s name: ” + item + “, Cost per item: ”

+ NumberFormat.getCurrencyInstance().format(perCost) + “, Quantity: ” + quantity + “, Cost: ”

+ NumberFormat.getCurrencyInstance().format(quantity * perCost) + “\n”);

}

clientOrder.append(“Order Total: ” + NumberFormat.getCurrencyInstance().format(total) + “\n”);

/* add the string buffer into global StringBuffer array (finalMessage). */

this.finalMessage[orderNum] = clientOrder;

}

/**

* This method creates a StringBuffer appended with summary of all orders, it

* then inserts this StringBuffer into global StringBuffer array (finalMessage).

* NOTE: must be called after processorOrder, or itemsQuantity will be empty

*/

public void processSummary() {

Double total = 0.0;

StringBuffer message = new StringBuffer();

message.append(“***** Summary of all orders *****\n”);

while (!itemsMap.isEmpty()) {

Map.Entry<String, Data> entry = itemsMap.pollFirstEntry();

int quantity = entry.getValue().getQuantity();

if (quantity != 0) { // only append if necessary

String item = entry.getKey();

Double perCost = entry.getValue().getSingleCost();

total += quantity * perCost;

message.append(“Summary – Item’s name: ” + item + “, Cost per item: ”

+ NumberFormat.getCurrencyInstance().format(perCost) + “, Number sold: ” + quantity

+ “, Item’s Total: ” + NumberFormat.getCurrencyInstance().format(quantity * perCost) + “\n”);

}

}

message.append(“Summary Grand Total: ” + NumberFormat.getCurrencyInstance().format(total) + “\n”);

finalMessage[finalMessage.length – 1] = message;

}

}

/**

* This class is created for faster incrementation of values in TreeMap AND

* better store of price per item

*/

class Data {

privateint quantity;

private Double singleCost;

public Data(int quantity, Double singleCost) {

this.quantity = quantity;

this.singleCost = singleCost;

}

publicintgetQuantity() {

returnthis.quantity;

}

public Double getSingleCost() {

returnthis.singleCost;

}

public void increment() {

this.quantity++;

}

} 

OrdersProcessor.java 

package processor;

importjava.io.BufferedWriter;

importjava.io.FileWriter;

importjava.io.IOException;

importjava.util.ArrayList;

importjava.util.Scanner;

public class OrdersProcessor {

public static void main(String[] args) {

/* Print user configuration in console. */

Scanner input = new Scanner(System.in);

System.out.print(“Enter item’s data file name: “);

String itemName = input.nextLine();

System.out.print(“Enter ‘y’ for multiple threads, any other character otherwise: “);

String isMultiThreaded = input.nextLine();

booleanisInvalid;

intnumOrders = 0;

do {

try {

System.out.print(“Enter number of orders to process: “);

numOrders = Integer.parseInt(input.nextLine());

isInvalid = false;

} catch (NumberFormatException e) {

System.out.println(“Invalid Input”);

isInvalid = true;

}

} while (isInvalid);

System.out.print(“Enter order’s base filename: “);

String baseName = input.nextLine();

System.out.print(“Enter result’s filename: “);

String resultsName = input.nextLine();

input.close();

/* Process order */

OrderProcessor order = new OrderProcessor(baseName, itemName, numOrders);

longstartTime = System.currentTimeMillis();

if (isMultiThreaded.equalsIgnoreCase(“y”)) {

/* Using multithread. */

ArrayList<Thread>threadList = new ArrayList<Thread>();

for (int i = 1; i <= numOrders; i++) { // start the threads

Thread t = new Thread(new MyThread(order, i));

threadList.add(t);

t.start();

}

for (Thread i : threadList) { // join the threads

try {

i.join();

} catch (InterruptedException e) {

e.printStackTrace();

}

}

order.processSummary(); // add Summary of All Orders

} else {

/* Not using multithread. */

for (int i = 1; i <= numOrders; i++) {

order.processorOrder(i);

}

order.processSummary(); // add Summary of All Orders

}

/* Write into results file. */

StringBufferfinalMessage = new StringBuffer();

StringBuffer[] messageList = order.getMessage();

for (int i = 1; i <messageList.length; i++) {

finalMessage.append(messageList[i]);

}

try {

BufferedWriterresultFile = new BufferedWriter(new FileWriter(resultsName));

resultFile.write(finalMessage.toString());

resultFile.close();

} catch (IOException e) {

e.printStackTrace();

}

/* Print last bit of information onto console */

longendTime = System.currentTimeMillis();

System.out.println(“Processing time (msec): ” + (endTime – startTime));

System.out.println(“Results can be found in the file: ” + resultsName);

}

}

classMyThread implements Runnable {

privateOrderProcessor order;

privateintorderNum;

publicMyThread(OrderProcessor order, intorderNum) {

this.order = order;

this.orderNum = orderNum;

}

public void run() {

order.processorOrder(orderNum);

}

} 

Table Runtime

Number of Orders Single-Thread Runtime (msec) Multi-Thread Runtime (msec)
100 3266 2523
200 5909 3645
300 8636 6060
400 11382 8121
500 14497 9810

Explanation

Sampled using Thinkpad W540, Intel Core i7-4910HQ @2.9GHz, with 8.00 GB of RAM. Operating System: Windows 10.Each data point has a sample size of 1.

It is obvious from the table and the graph that multithreading is superior in terms of runtime efficiency. Although it should be noted that with small runtime (e.g. example1, example2, and example3, with runtime of around 10 msec), it is hard to distinguish the difference since efficiency fluctuates. It should also be noted that since a critical part of the code was placed under synchronization, the benefit of multithreading was not fully demonstrated. 

PublicTests.java 

package tests;

importjava.io.BufferedReader;

importjava.io.File;

importjava.io.FileNotFoundException;

importjava.io.FileReader;

importjava.util.Scanner;

importjunit.framework.TestCase;

import processor.*;

public class PublicTests extends TestCase {

public void testSys1() throws Exception, Throwable {

runProgramWithInput(“pubTest1.txt”);

}

public void testSys2() throws Exception, Throwable {

runProgramWithInput(“pubTest2.txt”);

}

public void testSys3() throws Exception, Throwable {

runProgramWithInput(“pubTest3.txt”);

}

public void testSys4() throws Exception, Throwable {

runProgramWithInput(“pubTest4.txt”);

}

public void testSys5() throws Exception, Throwable {

runProgramWithInput(“pubTest5.txt”);

}

/**

* Executes a run of the OrdersProcessor program by reading the data

* in the specified file using input redirection.  The file inputFileName

* has the item’s data file, whether multiple threads will be used,

* number of orders, base file name for the orders, and the

* result file name.

*

* @paraminputFilename

* @throws Exception

* @throws Throwable

*/

private void runProgramWithInput(String inputFilename) throws Exception, Throwable {

/* Retrieving the name of the results file */

String resultsFilename = getResultsFilename(inputFilename);

String officialResultsFilename = resultsFilename + “.expected.txt”;

/* Deleting results file (in case it exists) */

File file = new File(resultsFilename);

file.delete();

/* Actual execution of the test by using input redirection and calling

/* OrdersProcessor.main(null) */

TestingSupport.redirectStandardInputTo(inputFilename);

OrdersProcessor.main(null);

/* Adding a specified string to results to avoid hard coding of results */

TestingSupport.appendStringToFile(resultsFilename, TestingSupport.hardCodingPrevention);

/* Ignore this method call. We use it to generate results for the submit server */

TestingSupport.generateOfficialResults(resultsFilename, officialResultsFilename);

/* Checking if we got the right results */

assertTrue(TestingSupport.sameContents(resultsFilename, officialResultsFilename));

}

/* Retrieves the name of the file to be used to store results */

private static String getResultsFilename(String filename) throws FileNotFoundException {

Scanner scanner = new Scanner(new BufferedReader(new FileReader(filename)));

for (int i = 1; i <= 4; i++) { // Throwing away first four items

scanner.next();

}

String name = scanner.next();

scanner.close();

return name;

}

/* This space available for rent */

/* When I grow up I want to be a Ruby program */

} 

TestingSupport.java 

package tests;

importjava.io.BufferedReader;

importjava.io.BufferedWriter;

importjava.io.ByteArrayOutputStream;

importjava.io.FileInputStream;

importjava.io.FileNotFoundException;

importjava.io.FileOutputStream;

importjava.io.FileReader;

importjava.io.FileWriter;

importjava.io.IOException;

importjava.io.InputStream;

importjava.io.OutputStream;

importjava.io.PrintStream;

importjava.util.Scanner;

importjava.util.StringTokenizer;

importjava.io.File;

importjavax.swing.JOptionPane;

public class TestingSupport {

/**

* Feel free to use the correctResults method while developing your own tests.

* Notice that if you define text files with some expected results, the text

* files must be named starting with “studentTest” and ending with the .txt extension.

* If you don’t name files this way, then the submit server will generate an authorization error.

* @param filename

* @param results

* @return true if the contents of the file corresponds to those in results

*/

public static booleancorrectResults(String filename, String results) {

String officialResults=””;

try {

BufferedReader fin = new BufferedReader(new FileReader(filename));

String line;

while ((line = fin.readLine()) != null) {

officialResults += line + “\n”;

}

fin.close();

}catch (IOException e) {

System.out.println(“File opening failed.”);

return false;

}

results = removeBlanks(results);

officialResults = removeBlanks(officialResults);

if (results.equals(officialResults)) {

return true;

}

return false;

}

public static booleansameContents(String firstFile, String secondFile) {

try {

if (removeBlanks(fileData(firstFile)).equals(removeBlanks(fileData(secondFile)))) {

return true;

}

} catch (IOException e) {

e.printStackTrace();

return false;

}

return false;

}

public static String fileData(String fileName) throws IOException {

StringBufferstringBuffer = new StringBuffer();

FileReaderfileReader = new FileReader(fileName);

BufferedReaderbufferedReader = new BufferedReader(fileReader);

Scanner fileScanner = new Scanner(bufferedReader);

while (fileScanner.hasNextLine())

stringBuffer.append(fileScanner.nextLine());

fileScanner.close();

returnstringBuffer.toString();

}

public static String removeBlanks(String src) {

return normalize(src);

}

public static String normalize(String in) {

StringTokenizerst = new StringTokenizer(in);

String retVal = new String();

while (st.hasMoreTokens()) {

retVal += st.nextToken();

}

returnretVal;

}

public static booleanwriteToFile(String filename, String message) {

try {

FileWriter output = new FileWriter(filename);

output.write(message);

output.close();

} catch(IOException exception) {

System.out.println(“ERROR: Writing to file ” + filename + ” failed.”);

return false;

}

return true;

}

/**

* Redirects standard input to be fileName

* @paramfileName

*/

public static void redirectStandardInputTo(String fileName) {

InputStreammyInput = null;

try {

myInput = new FileInputStream(fileName);

}

catch(FileNotFoundException e) {

System.out.println(“File not found.”);

}

System.setIn(myInput);

}

/**

* Resets standard/input output streams to default

*/

public static void resetStandardInputOutput() {

System.setOut(System.out);

System.setIn(System.in);

}

/**

* Redirects standard output to returned stream

* After running a program call toString on stream to

* get output.

* @return stream

*/

public static ByteArrayOutputStreamredirectStandardOutputToByteArrayStream() {

ByteArrayOutputStreamnewOutput = new ByteArrayOutputStream();

PrintStreamprintStream = new PrintStream(newOutput);

System.setOut(printStream);

returnnewOutput;

}

/**

* Makes a file copy

*

*/

public static booleancopyfile(String sourceFileName, String targetFileName) {

File sourceFile = new File(sourceFileName);

if (!sourceFile.exists()) {

System.err.println(sourceFileName + ” does not exist.”);

return false;

}

try {

InputStreaminputStream = new FileInputStream(sourceFileName);

OutputStreamoutputStream = new FileOutputStream(targetFileName);

int n;

while ((n = inputStream.read()) != -1) {

outputStream.write(n);

}

inputStream.close();

outputStream.close();

} catch(Exception e) {

System.err.println(“In copyfile ” + e.getMessage());

return false;

}

return true;

}

/* Feel free to ignore the following variables and methods */

/* They are used to set up the submit server */

private static booleangenerateOfficialResultsFlag = false;

/* We use this string to prevent any hardcoding of results. */

public static final String hardCodingPrevention = “TERPSPUBLICTESTS”;

public static void generateOfficialResults(String resultsFilename, String officialResultsFilename) {

/* Official use only (we used this to generate official result) */

if (generateOfficialResultsFlag) {

/* Copying results to official results filename */

if (!TestingSupport.copyfile(resultsFilename, officialResultsFilename)) {

System.err.println(“File copying failed”);

} else {

JOptionPane.showMessageDialog(null, officialResultsFilename + ” created. “);

}

}

}

public static void appendStringToFile(String filename, String stringToAppend) {

boolean append = true;

try {

BufferedWriterbufferedWriter = new BufferedWriter(new FileWriter(

filename, append));

bufferedWriter.write(stringToAppend);

bufferedWriter.flush();

bufferedWriter.close();

} catch(Exception e) {

System.err.println(e.getMessage());

}

}

}