Merge branch 'cg_merges_20121210' into bfgminer

NEWS

@@ -1,5 +1,116 @@
 BFGMiner Version 2.10.0 - Future
 
+- Include prctl header for thread renaming to work.
+- Set tv_idle time if a pool is not active when input from the menu.
+- usb display message when device is in use/another cgminer
+- libztex: avoid the use of libusb_error_name()
+- minor unlikely zero pointer test
+- BeaverCreek doesn't like BFI INT patching.
+- Only stratum pools that are idle need to be kicked via cnx_needed.
+- mmq - abbreviate the temperature numbers
+- Do not do any setup if opt_api_listen is disabled in api.c.
+- usbutils.c uninitialised usbstat for non-primary mmqs
+- Only set the lagging flag for select_pool() on failed getwork if we're not in
+opt_fail_only mode.
+- libztex: in case the selectFpga() failed set the selected fpga to unknown
+- Modified windows-build.txt to update git instructions.
+- libztex: use a function for the twice called firmware reset code
+- libztex: removed an unused struct member (ztex->valid)
+- driver-ztex: support for broken fpga on a multifpga board
+- Set the pool lagging flag on startup to avoid it being shown initially, and
+only unset it once the maximum number of staged work items has been reached.
+- Avoid recursive locking of the stgd lock.
+- Return value of keep_sockalive is no longer used.
+- Remove dependency on mstcpip.h for windows build by making curl version >=
+7.25.0 mandatory on windows builds, and use curl functions for keepalive
+whenever possible instead.
+- Make main() the getwork scheduler once everything is set up, so that all app
+exits use the kill_work and quit paths.
+- ztex: more style and whitespace fixes
+- libztex: silenced another warning
+- Set successful connect to true on auth stratum to allow summary on exit from
+single stratum pool.
+- Only consider work stale for stratum of different job_id if it's not a share.
+- Increment version preempting changed version signifying different codebase to
+2.9
+- Hash_pop should signal further waiters on its own pthread conditional in case
+there are multiple waiters.
+- Check the job_id has not changed on stratum work when deciding if the work is
+stale as might occur across disconnections.
+- Perform pool_resus on getwork pool that generates work in getwork_thread.
+- Set pool lagging message for getwork pool that falls to zero staged in getwork
+thread.
+- Stage extra work when the primary pool is a getwork pool without rolltime.
+- Do not try to clean up twice if kill message is given.
+- Only recalculate total_staged in getwork thread if required.
+- Include the correct config header in libztex and include it before other
+includes.
+- Implement a completely new getwork scheduler. Stage all work from the one
+thread, making it possible to serialise all requests minimising the number of
+getworks requested or local work generated. Use a pthread conditional to wake up
+the thread whenever work is removed to generate enough work to stay above the
+watermark set by opt_queue. Remove all remnants of the old queueing mechanism,
+deleting the now defunct queued count.
+- libztex: fixed some warnings and removed some whitespaces
+- libztex: silenced some warnings
+- Remove all references to the now unused workio_cmd structure.
+- Remove the old workio command queue thread, replacing it with a kill
+conditional to exit the program.
+- Remove getwork command from workio_cmd queues and do them directly from
+queue_request.
+- Begin tearing down the old workio command queues by removing submit commands
+from there and submit them asynchronously via their own threads.
+- Update windows build instructions.
+- Set pool probed to true on successful authorisation with stratum to avoid it
+being pinged later with pool_getswork.
+- driver-ztex: libztex_setFreq() must be called before ztex_releaseFpga()
+- driver-ztex: changed two pairs of malloc()/memset() to calloc()
+- libztex: Read bitstream file in 2kb blocks with simpler and faster code
+- Added the binary versions of ztex_ufm1_15d4.ihx and ztex_ufm1_15y1.ihx
+- Trivial space removal.
+- libztex: Add firmware download support for ZTEX 1.15d and 1.15x
+- libztex: Factor out local version of libusb_get_string_descriptor_ascii()
+- Shut up some boring old cpu warnings.
+- Style changes.
+- Allow pool active to be called on stratum or disabled pools in the watchpool
+thread if the pool has not been probed.
+- libztex: Make log messages say bitstream when refering to bitstreams
+- libztex: Don't return error when a bitstream was already configured
+- libztex: Read bitstream file in 64kb blocks with simpler and faster code
+- libztex: Verify that the mining firmware is not a dummy firmware
+- libztex: Match mining firmware ZTEX descriptor against the dummy firmware
+- Combine shared padding into one char.
+- libztex: Start download sequence only after reading in the new firmware
+- libztex: Download mining firmware to all devices with dummy firmware
+- lock (most of) the threaded statistics updates
+- README stats don't add up
+- usbutils.c remove compiler warning
+- Make need connection return true if a pool is idle.
+- API add Best Share to summary
+- Check on creating new GBT work if the structures are up to date and update
+them as required rather than regularly.
+- Update windows build instructions.
+- Enable backup stratum connections for getwork when the primary pool doesn't
+have longpoll aka solo mining.
+- Check for correct absence of opt_fail_only in cnx_needed.
+- Remove unused variable.
+- The specification for stratum has been elaborated to say that a changed diff
+applies only to new work so do not retarget when submitting shares.
+- Use a variable length string array in submit_upstream_work to cope with
+massive GBT submissions.
+- API lock access to some summary statistics (and copy them)
+- Suspend stratum connections to backup pools when there is no requirement to
+potentially grab work from them.
+- Fix missing export for RenameThread.
+- enumerate the mining threadnames
+- MMQ avoid possible number overrun crashes
+- mmq usb v0.4 + api usb stats
+- setting the name of the threads for linux,freebsd,openbsd and osx code is
+borrowed from bitcoins util.c, so it is already tested
+- Don't show broken WU value with scrypt mining.
+- Style police.
+- Remove unused getwork times in getswork.
+- Fix readme wordwrap.
 - New --skip-security-checks option to allow miners to skip checks when it
 saves bandwidth
 - Skip stratum transaction download when there are no transactions

README

@@ -433,9 +433,8 @@ The number of hardware erorrs
 The utility defines as the number of shares / minute
 
 The BFGMiner status line shows:
- TQ: 1  ST: 1  DW: 0  GW: 301  LW: 8  GF: 1  NB: 1  AS: 0  RF: 1
+ ST: 1  DW: 0  GW: 301  LW: 8  GF: 1  NB: 1  AS: 0  RF: 1
 
-TQ is Total Queued work items.
 ST is STaged work items (ready to use).
 DW is Discarded Work items (work from block no longer valid to work on)
 GW is GetWork requested (work items from pools)

api.c

@@ -3365,16 +3365,16 @@ void api(int api_thr_id)
 	bool did;
 	int i;
 
-	mutex_init(&quit_restart_lock);
-
-	pthread_cleanup_push(tidyup, NULL);
-	my_thr_id = api_thr_id;
-
 	if (!opt_api_listen) {
 		applog(LOG_DEBUG, "API not running%s", UNAVAILABLE);
 		return;
 	}
 
+	mutex_init(&quit_restart_lock);
+
+	pthread_cleanup_push(tidyup, NULL);
+	my_thr_id = api_thr_id;
+
 	setup_groups();
 
 	if (opt_api_allow) {

driver-ztex.c

@@ -189,6 +189,9 @@ static int64_t ztex_scanhash(struct thr_info *thr, struct work *work,
 	bool overflow, found;
 	struct libztex_hash_data hdata[GOLDEN_BACKLOG];
 
+	if (thr->cgpu->deven == DEV_DISABLED)
+		return -1;
+
 	ztex = thr->cgpu->device_ztex;
 
 	memcpy(sendbuf, work->data + 64, 12);
@@ -364,7 +367,9 @@ static bool ztex_prepare(struct thr_info *thr)
 	if (libztex_configureFpga(ztex) != 0) {
 		libztex_resetFpga(ztex);
 		ztex_releaseFpga(ztex);
-		return false;
+		applog(LOG_ERR, "%s: Disabling!", thr->cgpu->device_ztex->repr);
+		thr->cgpu->deven = DEV_DISABLED;
+		return true;
 	}
 	ztex->dclk.freqM = ztex->dclk.freqMaxM+1;;
 	//ztex_updateFreq(thr);

libztex.c

@@ -104,6 +104,19 @@ enum check_result
 	CHECK_RESCAN,
 };
 
+static bool libztex_firmwareReset(struct libusb_device_handle *hndl, bool enable)
+{
+	uint8_t reset = enable;
+	int cnt = libusb_control_transfer(hndl, 0x40, 0xA0, 0xE600, 0, &reset, 1, 1000);
+	if (cnt < 0)
+	{
+		applog(LOG_ERR, "Ztex reset %d failed: %s", libusb_error_name(cnt), enable);
+		return 1;
+	}
+
+	return 0;
+}
+
 static enum check_result libztex_checkDevice(struct libusb_device *dev)
 {
 	FILE *fp = NULL;
@@ -249,14 +262,8 @@ static enum check_result libztex_checkDevice(struct libusb_device *dev)
 		goto done;
 	}
 
-	// reset 1
-	buf[0] = 1;
-	cnt = libusb_control_transfer(hndl, 0x40, 0xA0, 0xE600, 0, buf, 1,1000);
-	if (cnt < 0)
-	{
-		applog(LOG_ERR, "Ztex reset 1 failed: %s", libusb_error_name(cnt));
+	if (libztex_firmwareReset(hndl, true))
 		goto done;
-	}
 
 	for (i = 0; i < length; i+= 256) {
 		// firmware wants data in small chunks like 256 bytes
@@ -269,14 +276,8 @@ static enum check_result libztex_checkDevice(struct libusb_device *dev)
 		}
 	}
 
-	// reset 0
-	buf[0] = 0;
-	err = libusb_control_transfer(hndl, 0x40, 0xA0, 0xE600, 0, buf, 1,1000);
-	if (err < 0)
-	{
-		applog(LOG_ERR, "Ztex reset 0 failed: %s", libusb_error_name(err));
+	if (libztex_firmwareReset(hndl, false))
 		goto done;
-	}
 
 	applog(LOG_ERR, "Ztex device: succesfully wrote firmware");
 	ret = CHECK_RESCAN;
@@ -538,6 +539,7 @@ int libztex_selectFpga(struct libztex_device *ztex)
 		cnt = libusb_control_transfer(ztex->root->hndl, 0x40, 0x51, (uint16_t)number, 0, NULL, 0, 500);
 		if (unlikely(cnt < 0)) {
 			applog(LOG_ERR, "Ztex check device: Failed to set fpga with err %d", cnt);
+			ztex->root->selectedFpga = -1;
 			return cnt;
 		}
 		ztex->root->selectedFpga = number;
@@ -700,7 +702,6 @@ int libztex_prepare_device(struct libusb_device *dev, struct libztex_device** zt
 	newdev->usbbus = libusb_get_bus_number(dev);
 	newdev->usbaddress = libusb_get_device_address(dev);
 	sprintf(newdev->repr, "ZTEX %s-1", newdev->snString);
-	newdev->valid = true;
 	return 0;
 }
 
@@ -781,7 +782,6 @@ int libztex_scanDevices(struct libztex_dev_list*** devs_p)
 
 		ztex->bitFileName = NULL;
 		ztex->numberOfFpgas = -1;
-		ztex->valid = false;
 
 		err = libztex_prepare_device(list[usbdevices[i]], &ztex);
 		if (unlikely(err != 0)) {

libztex.h

@@ -45,7 +45,6 @@ struct libztex_device {
 	pthread_mutex_t	mutex;
 	struct libztex_device *root;
 	int16_t fpgaNum;
-	bool valid;
 	struct libusb_device_descriptor descriptor;
 	libusb_device_handle *hndl; 
 	unsigned char usbbus;

miner.c

@@ -3173,8 +3173,11 @@ static void recruit_curl(struct pool *pool)
 {
 	struct curl_ent *ce = calloc(sizeof(struct curl_ent), 1);
 
+	if (unlikely(!ce))
+		quit(1, "Failed to calloc in recruit_curl");
+
 	ce->curl = curl_easy_init();
-	if (unlikely(!ce || !ce->curl))
+	if (unlikely(!ce->curl))
 		quit(1, "Failed to init in recruit_curl");
 
 	list_add(&ce->node, &pool->curlring);
@@ -5468,8 +5471,8 @@ static bool cnx_needed(struct pool *pool)
 	if (pool_strategy == POOL_LOADBALANCE)
 		return true;
 
-	/* Idle pool needs something to kick it alive again */
-	if (pool->idle)
+	/* Idle stratum pool needs something to kick it alive again */
+	if (pool->has_stratum && pool->idle)
 		return true;
 
 	/* Getwork pools without opt_fail_only need backup pools up to be able
@@ -7143,8 +7146,10 @@ void add_pool_details(struct pool *pool, bool live, char *url, char *user, char
 
 	/* Test the pool is not idle if we're live running, otherwise
 	 * it will be tested separately */
-	if (live && !pool_active(pool, false))
+	if (live && !pool_active(pool, false)) {
+		gettimeofday(&pool->tv_idle, NULL);
 		pool->idle = true;
+	}
 }
 
 #ifdef HAVE_CURSES
@@ -7789,6 +7794,7 @@ int main(int argc, char *argv[])
 					continue;
 
 				if (pool_active(pool, false)) {
+					pool_tset(pool, &pool->lagging);
 					pool_tclear(pool, &pool->idle);
 					if (!currentpool)
 						currentpool = pool;
@@ -8082,7 +8088,7 @@ retry:
 			push_curl_entry(ce, pool);
 			++pool->seq_getfails;
 			pool_died(pool);
-			next_pool = select_pool(true);
+			next_pool = select_pool(!opt_fail_only);
 			if (pool == next_pool) {
 				applog(LOG_DEBUG, "Pool %d json_rpc_call failed on get work, retrying in 5s", pool->pool_no);
 				sleep(5);
@@ -8092,7 +8098,8 @@ retry:
 			}
 			goto retry;
 		}
-		pool_tclear(pool, &pool->lagging);
+		if (ts >= max_staged)
+			pool_tclear(pool, &pool->lagging);
 		if (pool_tclear(pool, &pool->idle))
 			pool_resus(pool);
 
@@ -8103,4 +8110,3 @@ retry:
 
 	return 0;
 }
-

ocl.c

@@ -814,8 +814,7 @@ build:
 		    strstr(name, "Antilles" ) ||
 		    strstr(name, "Wrestler" ) ||
 		    strstr(name, "Zacate" ) ||
-		    strstr(name, "WinterPark" ) ||
-		    strstr(name, "BeaverCreek" ))
+		    strstr(name, "WinterPark" ))
 			patchbfi = true;
 	} else
 		applog(LOG_DEBUG, "cl_amd_media_ops not found, will not set BITALIGN");

usbutils.c

@@ -714,6 +714,10 @@ bool usb_init(struct cgpu_info *cgpu, struct libusb_device *dev, struct usb_find
 
 	cgusb = calloc(1, sizeof(*cgusb));
 	cgusb->found = found;
+
+	cgusb->bus_number = libusb_get_bus_number(dev);
+	cgusb->device_address = libusb_get_device_address(dev);
+
 	cgusb->descriptor = calloc(1, sizeof(*(cgusb->descriptor)));
 
 	err = libusb_get_device_descriptor(dev, cgusb->descriptor);
@@ -737,7 +741,6 @@ bool usb_init(struct cgpu_info *cgpu, struct libusb_device *dev, struct usb_find
 			default:
 				applog(LOG_ERR, "USB init, open device failed, err %d", err);
 		}
-
 		goto dame;
 	}
 
@@ -751,8 +754,15 @@ bool usb_init(struct cgpu_info *cgpu, struct libusb_device *dev, struct usb_find
 
 	err = libusb_set_configuration(cgusb->handle, found->config);
 	if (err) {
-		applog(LOG_DEBUG, "USB init, failed to set config to %d, err %d",
-			found->config, err);
+		switch(err) {
+			case LIBUSB_ERROR_BUSY:
+				applog(LOG_WARNING, "USB init, %s device %d:%d in use",
+						found->name, cgusb->bus_number, cgusb->device_address);
+				break;
+			default:
+				applog(LOG_DEBUG, "USB init, failed to set config to %d, err %d",
+						found->config, err);
+		}
 		goto cldame;
 	}
 
@@ -797,8 +807,6 @@ bool usb_init(struct cgpu_info *cgpu, struct libusb_device *dev, struct usb_find
 		goto cldame;
 	}
 
-	cgusb->bus_number = libusb_get_bus_number(dev);
-	cgusb->device_address = libusb_get_device_address(dev);
 	cgusb->usbver = cgusb->descriptor->bcdUSB;
 
 // TODO: allow this with the right version of the libusb include and running library
@@ -1061,6 +1069,9 @@ static void newstats(struct cgpu_info *cgpu)
 void update_usb_stats(__maybe_unused struct cgpu_info *cgpu)
 {
 #if DO_USB_STATS
+	if (cgpu->usbstat < 1)
+		newstats(cgpu);
+
 	// we don't know the device_id until after add_cgpu()
 	usb_stats[cgpu->usbstat - 1].device_id = cgpu->device_id;
 #endif

util.c

@@ -10,7 +10,6 @@
  * any later version.  See COPYING for more details.
  */
 
-#define _GNU_SOURCE
 #include "config.h"
 
 #ifdef WIN32
@@ -36,6 +35,9 @@
 # include <pthread_np.h>
 #endif
 #ifndef WIN32
+# ifdef __linux
+#  include <sys/prctl.h>
+# endif
 # include <sys/socket.h>
 # include <netinet/in.h>
 # include <netinet/tcp.h>

windows-build.txt

@@ -171,6 +171,8 @@ To test if it is working, open a MinGW shell and type the following:
 
 If you simply just want to update the source after you have already cloned, type:
   git pull
+"git pull" did not work for me. Try the following which does the same thing:
+  git fetch && git merge FETCH_HEAD
 
 Now you can get the latest source directly from github.